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(57) Abstract 

A system and method for editing distal images in three dimensions 
includes a computer (130) for storing a digital image of an object (112, 
156) and a background (1 14), as well as at least one additional background 
image (157). Based upon the difference between the hues of the edge (146) 
of the object and the surrounding background (1 14) and a predetermined 
hue <^erence, die computer (130) locates the edge of the object (112, 156) 
and removes portions of the image (i.e., die background (114)) that are 
outside the edge (146). Then, the object (112, 156) can be combined widi 
a preselected one of the other background images (157) so as to form a 
composite image (159). Components of the preselected background image 
(157) are assigned relative positions in the X-Y plane, and are also assigned 
a value defining their location in one of a plurality of layers which form 
the Z dimension of die image. The object (1 12, 156) to be combined widi 
the background (157) is also assigned a value dining its location in at 
least one of those layers. In another embodiment of the invention, colors 
of either a digital or video image can be selectively assigned to a mask. 
The colors can be of die entire image or from a selected area (1370) of die 
image. Color manipulation can then be performed on just the colors of die 
image (1352, 13S4, 1356) defined by the noask. The mask may be used 
with the entire image, with a selected area of die image (1350), or with 
objects. Alteniatively, the colors of the image defined by an inverted mask 
are affected by color manipulation. 
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IMAGE EDITING SYSTEM INCLUDING MASKING CAPABILITY 



Background of the Invention 
Field of the Invention 

The present invention relates to image processing and, 
more particularly, to systems for editing digital images. 
Background of the Technology 

A large number of applications require combining one 
video image with another video image, i.e., a television 
broadcast of a weather person in front of weather maps during 
a weather report, so as to produce a composite video image . 
One well-known technique for producing composite video images 
is commonly referred to as " chroma- key " . The chroma-key 
technique is so named because it uses the chroma or color 
information portion of a television signal as a "key" to 
control the formation of a composite image. A chroma-key 
device is essentially a video multiplexer which selects a 
video input signal by feeding the chroma signal from one of 
the two video inputs to decision logic. 

In the case of the above-mentioned weather report 
application, a television camera is directed at a weather 
reporter standing in front of a vertical sheet, called a 
matte, which has a predetermined blue hue or color shade. Hue 
is the dimension of color that is referred to a scale of 
perceptions ranging from red through yellow, green, and blue, 
and circularly back to red. The image of the weather reporter 
and matte foreground source is provided to the chroma-key 
device, which is programmed to select all of the video signals 
received from the camera, except those video signals that 
represent portions of the image having the predetermined blue 
hue. Thus, the chroma-key device effectively separates the 
image of the reporter from the image of the matte. 

At the same time that the previously described operation 
takes place, a video image of a weather map or satellite view. . 
of the earth, either of which may be superimposed with city 
names, high/low temperatures, and the like, is provided to the 
chroma-key device as a background source. The decision logic 
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of the chroma key device selects the background source as 
video output wherever a blue hue is detected in the foreground 
source and presents the resulting background and foreground as 
a composite picture on a studio monitor that can be viewed by 
5 the reporter. The reporter can then point to positions on the 

matte that correspond to geographic locations on the 
background source and the viewer of the television program 
sees a composite image of a reporter and a weather map. Such 
a composite image is the desired output of a chroma-key 
10 device. 

However, as was noted in U.S. Patent No. 4,811,084 to 
Belmares-Sarabia, et al . , a major disadvantage of a chroma-key 
system is that false keys can be produced. For example, 
weather reporters wearing blue or striped suits, and even blue 

15 eyes, may cause a chroma-key system to produce an incorrectly 

spliced composite. Also, a chroma-key device is used too 
large a distance between the reporter and the matte causing 
reflections resulting from false keying, hence restraining the 
movements of the reporter. 

20 To overcome the problems inherent in chroma -keying, the 

Belmares-Sarabia, et al., patent discusses a device for video 
color detection that does not depend on a single color for 
keying. For example, such a device is also said to 
discriminate among similar hues by limiting the bandwidth of 

25 the hues and/or hue and saturation combinations that can be 

recognized by the device. 

The device disclosed in Belmares-Sarabia, et al . , uses an 
analog process to multiplex television signals. Analog 
processes, however, are not as versatile in combining images 

30 as are digital processes, which can be programmed to apply 

sophisticated image processing algorithms to a digitized image 
so as to alter or edit an image. Thus, it would be an 
advancement in the technology to provide a digital image 
editing system which can strip a digital image of an object 

35 from a background and combine the digital object with a 

different digital background or backgroiinds (composite) 
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without suffering from the above-mentioned problems and 
limitations . 

Other ways of producing a composite image include image 
editing software programs running on a Macintosh® computer 
5 available from Apple Computer, Inc., or a PC type compatible 

computer available from IBM and other companies. These 
programs are exemplified by Picture Publisher® produced by 
Micrografx, Inc. for the PC and Adobe Photoshop™ produced by 
Adobe systems Incorporated for the Macintosh. Picture 

10 publisher is a registered trademark of Micrografx, Inc. Adobe 

Photoshop is a trademark of Adobe Systems Incorporated. 
Macintosh is a registered trademark of Apple Computer, Inc. 

These programs enable the user to place one object image 
in front of a background scene and to cut and remove the 

15 object image. However, these programs are limited to working 

with one object only. These programs cannot build a set of 
layers of objects and backgrounds and allow the user to move 
an object to different depths or layers into the composite 
image. In other words, a person could not be placed behind a 

20 fence but in front of a house of the background scene 

simultaneously. Thus, it would be an advancement in 
technology to provide a system which could place an object in 
front or behind any other object or the background at any 
desired depth into the composite image. It would also be 

25 desirable to make part of an object which was moved into a 

particular layer to be transparent based on a desired 
attribute of the object. For example, one attribute of the 
object is hue, which is the perceived color shade. The leaves 
of a tree, which have a particular hue range of green, could 

30 be set to be transparent rather than opaque. Thus, the sky 

would then be seen between the branches of the tree, wherein 
previously, the leaves had blocked the sky. 

When producing a composite image involving the face of a 
person, one may desire to remove the original person's face 
35 and replace it with the face of another person. Frequently 

the two faces will not be the same size, e.g., one face is a 
closeup and the other is not, and the person trying to make 
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the coti^osite image will have to reduce or enlarge the 
replacement face to fit. This may involve numerous trials to 
achieve a pleasing appearance. Thus, it would be desirable to 
provide a system which can automatically size the replacement 
5 face to provide a natural appearance without a trial and error 

process. 

Another aspect to consider when replacing one face for 
another is the coloration of the skin. The person making the 
composite image may be placing a fair complexioned face to 

10 replace a dark complexioned face. The rest of the body, such 

as hands, arms and legs, may be visible in the original image. 
The composite would therefore not appear natural after the 
replacement was done. Thus, it would be desirable to provide 
a system which can automatically match the skin tone of the 

15 replacement face to that of the original face without any 

manual intervention to provide a natural appearance for the 
resultant composite image . 

A further aspect to consider when replacing one face for 
another in creating a composite image is the positioning of 

20 the replacement face . This positioning may involve an 

iterative process to try different placements to achieve a 
pleasing and natural appearance. Thus, it would be desirable 
to provide a system which can automatically position the 
replacement face at the appropriate location to produce a 

25 pleasing appearance for the resultant composite image. 

It is often the case that an object is imaged under one 
lighting condition and is then overlaid on a background that 
was imaged under another lighting condition . Consequently, 
the composite image may look artificial. Thus, it would be a 

30 further advantage if the editing system could establish the 

same lighting conditions, or "gamma", for the entire composite 
image . For example, it may be desirable to have an object 
that was imaged under fluorescent light inserted into a 
background that was imaged under full daylight and have the. 

3 5 composite image maintain the same lighting condition . The 

lighting condition of the coirposite image could even be a 
third condition such as moonlight. 
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While editing a color image, a person may wish to perform 
a particular color manipulation or operation on only a certain 
color or group of colors rather than all the colors of the 
image. This certain color or group of colors defines a mask. 
5 Alternatively, one may wish to perform a particular color 

manipulation or operation on all colors of an image except a 
certain color or group of colors. This is accomplished by 
taking all the colors of the image except those defined by the 
first group of colors through an inversion process. Present 

10 technology allows use of colors, but only for an area of the 

image, and then using only one color at a time. An area is a 
portion of or the entire image used as a background or base 
image, and is considered to be a single layer. Thus, it would 
be an advance in the technology to provide multi-color mask 

15 technology for use in image manipulation. These masks would 

apply to either the entire image, to a selected area of the 
image, or to objects of the image. A further advance would 
allow color image manipulation using masks on a series or 
sequence of related images. 

20 Accordingly, a need exists to provide a digital image 

editing system which can separate the digital image of an 
object from a background against which the object was imaged. 
It is a further purpose of the present invention to provide a 
digital image editing system which can automatically size, 

25 position, and layer the digital image of a replacement object 

or multiple objects into a predetermined background at a 
desired depth, and then match the lighting conditions of the 
replacement object with one or more original objects and the 
background, and to provide a digital image editing system that 

30 is easy to implement and cost-effective to use. A need also 

exists to provide color masking capability to include 
simultaneous multiple colors and the choice of their usage 
with objects, areas, or the entire image. 



35 



Summary of the Invention 
The present invention satisfies the above-mentioned needs 
and includes a system and method for selectively combining 
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digital images. The system includes a computer, which is 
connected to a device, such as a video camera, that can 
generate a signal representing an image. An object, for 
example, a human, is positioned in front of the video camera, 
5 and the video camera generates a signal representing the 

object and the background that is behind the object, 

Accordingly, the signal from the camera includes an 
object component representative of the image of the object and 
a background component representative of the image of the 
.0 background. The object has an edge, and the computer detects 

the edge of the object and separates portions of the image 
that are outside the edge of the object (i.e., the background 
component) from portions of the image that are inside the edge 
(i.e., the object component). 
5 In one embodiment, the background has a single continuous 

hue, and based upon the difference in hue between the object 
and background, the computer determines the location of the 
edge of the object. Based on this determination, the computer 
removes the background component from the signal . 
0 More particularly, the video camera produces through 

digitization a digital signal that is conprised of a plurality 
of pixels that are arranged in rows. Each pixel has a hue 
gamma and each hue gamma has a corresponding numeric value 
which represents how light or dark the hue is. The computer 
5 determines the location of the edge of the object by first 

ascertaining the numeric value of the hue gamma of a first 
pixel that is located at the end of the top row of the video 
window (i.e., is located at the periphery of the video image) . 
The computer also determines the niimeric value of the hue 
0 gamma of an adjacent second pixel in the row, and then 

coTt^ares the difference between the gammas to a predetermined 
difference . 

When the difference between the hue gamma of the first 
pixel and the hue gamma of the second pixel is less than the. 
5 predetermined difference, the computer compares the hue gamma 

of the second pixel with the hue gamma of a third pixel that 
is located in the same row as the first and second pixels and 
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is adjacent to the second pixel, and so on. When the 
difference between amy two pixels exceeds the predetermined 
value, this indicates the presence of a portion of the edge of 
the object. The computer then stores the location of the edge 
5 portion in memory, and then repeats the process described 

above for the row immediately below the top row of pixels. If 
desired, the computer can simultaneously perform a process 
like the one described above, but working from another 
direction, such as upwardly from the bottom row of pixels. 

10 The conputer continues the process, i.e., working 

inwardly from the boundary of the video window in the pixel - 
by-pixel comparisons, until the entire edge of the object has 
been mapped. After mapping the edge of the object, the 
computer is ready to "strip" (i.e., remove) the background 

15 component from the object component by setting all background 

pixels to a preselected transparent value. 

After stripping away the background image from the object 
image, the computer can, if desired, integrate the object 
image into a preselected background image that is different 

20 than the background against which the object was imaged. More 

particularly, the computer can have one or more preselected 
background images stored in the memory of the computer. The 
computer can selectively digitally combine the object 
component of the signal with multiple signals which are 

25 representative of multiple layers of these stored backgrounds, 

together with optional text entered by an operator or user of 
the system and present the composite image on a video display. 
Thus, the image of the object can be combined, if desired, 
with a preselected background that is different from the 

30 background against which the object was imaged, user text 

added, if desired, and the composite image displayed. 

Stated differently, the conrputer can essentially function 
as an image combiner by stripping, from a digital video 
signal, the image of an object from the image of the 

35 background in front of which the object was positioned. The 

computer then combines the image of the object with a 
preselected image. Specifically, stored in the memory of the 
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computer is a two-dimensional digital image representation of 
a three-dimensional field of view, with components of the 
digital image assigned layer values to define their position 
in the three-dimensional field of view. The conputer then 
5 blends the object into the selected background at the desired 

layer and X-Y position, so that the object appears to be 
integrated into the three-dimensional field of view. 

Preferably, to further blend the object into the 
preselected image, the computer averages the hue of edge of 
10 the object and the hue of the portion of the preselected 

background that is contiguous to the edge. The computer then 
adjusts the hue of the edge of the object to equal the 
averaged hue. 

Additionally, the computer of the present invention can 

15 adjust the gamma of one or both of the object and the 

preselected stored background, to make the integrated image 
appear as if the object was imaged under the same lighting 
conditions as the preselected stored background. For example, 
the computer can ascertain the gamma of the preselected 

20 background, and then adjust the numeric value of the hue of 

the pixels that make up the image of the object as appropriate 
to make the object appear as though it were imaged under the 
same lighting conditions under which the preselected 
background was imaged. 

25 The system of the present invention can also include a 

video printer electrically connected to the computer for 
generating a picture of the object. Also, the system may 
include a currency acceptor which is operably engaged with the 
computer for accepting currency and activating the system in 

30 response to the insertion of currency into the acceptor. 

Thus, the. system may, in one application, be used in 
conjimction with electrical imaging booths that are installed 
in publ i c places for use by the piiblic. 

The system of the present invention automatically sizes 

3 5 the obj ect image to naturally integrate into the composite 

image. The computer compares the original background object 
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size to that of the replacement object and adjusts the size, 
if necessary, of the replacement object - 

The system of the present invention automatically 
positions the object image to naturally integrate into the 
5 con^osite image. The computer utilizes the address of a 

predetermined location on the original background object and 
transfers that address to a predetermined location of the 
replacement object. 

Traditional methods for image editing use area selection 

10 to identify the pixels of the image that are to be changed. 

The present invention allows an alternate means of color to be 
used for selecting the pixels of the image to be changed. The 
present invention includes a system and method for choosing 
the existing colors of an image that either will or will not 

15 be subject to further color mcinipulation. The user can choose 

a single color or a plurality of colors to be simultaneously 
included in a mask. This process of applying a mask involves 
two types of masks. The first type of mask, called the 
original mask, includes the colors selected by the user. The 

2 0 second type of mask, called the inverted mask, includes all 

colors of the image except those of the original mask. A mask 
defines the colors which will be affected by further color 
manipulations. The manipulation applies to all colors, in the 
mask simultaneously. This mask can be in one of three 

25 conditions: no colors, some colors of the image, or all colors 

of the image. An inverted mask defines all of the colors in 
the image except those of the original mask (before inversion) 
which will be affected by further color manipulations. The 
mask may be used with the entire image, with a selected area 

30 of the image, or with objects. The mask therefore identifies 

specific pixels within the image, selected area or object that 
are subject to further manipulation, which includes a subset 
of the area , brush , and ob j ec t opt ions , e.g., blur . Upon 
completion of the blur operation, only the pixels of the area. 

3 5 or object that match the color defined by the mask will be 

blurred. Thus the advanced masking capability, through color 
selection rather than area selection, enables the user to 
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quickly and efficiently modify or enhance the appearance of an 
image to desired goal. As an enhancement of the masks, each 
mask type may be applied to a series of related images for 
color manipulation. 
5 In one aspect of the invention, in a computer system 

having a memory, there is a method of editing an image having 
a plurality of pixels, each* pixel having a color, comprising 
the steps of selecting a plurality of colors from among those 
present in the image to form a set of selected colors, and 
10 manipulating only those pixels having a color defined in the 

set of selected colors so as to change a visual feature of the 
image * 

In another aspect of the present invention, in a computer 
system having a memory, there is a method of editing an image 

15 having a plurality of pixels, each pixel having a color, 

comprising the steps of identifying a color of each pixel of 
the image, storing data representing each identified color in 
an original first list on the memory of the computer system, 
identifying a selected color of the image, deleting the data 

20 representing the selected color from the first list, and 

manipulating pixels of the image having colors of a 
preselected relationship to the colors identified in the first 
list so as to change a visual feature of the image. 

In yet another aspect of the present invention, in a 

25 computer system having a memory, there is a method of creating 

a color mask for an image, comprising the steps of providing 
an image having a plurality of pixels, each pixel having a 
color, selecting a plurality of colors from among those 
present in the image to form a set of selected colors, and 

30 storing data representing the colors from the set of selected 

colors in a list on the memory of the computer system, to 
thereby define the mask- 

In another aspect of the present invention, in a computer 
system having a memory, there is a method of creating an 

35 inverted color mask for an image, comprising the steps of 

providing an image having a plurality of pixels, each pixel 
having a color, identifying the color of each pixel of the 
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image to form a set of identified colors, selecting a color of 
the image, and deleting the selected color from the colors in 
the set of identified colors, thereby defining an inverted 
mask. 

5 In yet another aspect of the present invention, there is 

a system for editing an image having a plurality of pixels, 
each pixel having a color, comprising a computer having a 
memory, means for providing the image, means for identifying 
a color of each pixel of the image, means for storing data 

10 representing each identified color in a first list on the 

memory of the computer, means for identifying a selected color 
of the image, means for deleting the data representing the 
selected color from the first list, and means for manipulating 
pixels of the image having colors corresponding to the colors 

15 identified in the first list so as to change a visual feature 

of the image. 

In another aspect of the present invention, in a computer 
system, there is a method of recoloring an object having a 
plurality of pixels, comprising the steps of providing a 

20 background image having a plurality of pixels, selecting an 

area from the background image, determining a hue of a 
selected pixel in the selected area, repeating the determining 
step until the hue of all pixels in the selected area has been 
determined, determining an average hue of all the pixels in 

25 the selected area, selecting an object to be recolored, and 

changing the hue of the selected object to the calculated 
average hue . 

In yet another aspect of the present invention, in a 
computer system, there is a method of recoloring an object 

30 having a plurality of pixels, comprising the steps of 

providing a background image having a plurality of pixel^s, 
selecting am area from the background image, identifying the 
selected area as a first object, determining a hue of a 
selected pixel of the first object, repeating the determining 

35 step until the hue of all pixels of the first object has been 

determined, determining an average hue of all the pixels of 
the first object, identifying a second object to be recolored. 
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and changing the hue of the second object to the average hue 
of the first object. 

These and other objects and features of the present 
invention will become more fully apparent from the following 
5 description and appended claims taken in conjunction with the 

accompanying drawings, in which like numerals refer to like 
parts . 
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Brief Description of the Drawings 
Figure 1 is a perspective view of a presently preferred 
embodiment of the imaging system of the present invention, 
with portions cut away and shown in phantom, and with a human 
stibject sitting in front of a background of a single hue; 

Figure 2 is a block diagram illustrating the components 
15 of the imaging system of Figure 1; 

Figures 3a, 3b and 3c are a top-level flow diagram of the 
imaging system, used in conjunction with the monochrome 
background of Figure 1; 

Figures 4a, 4b, 4c, 4d, 4e, 4f , 4g, and 4h are a sequence 
20 of images exemplary of those produced by the imaging system in 

performing the process steps shown in Figure 3; 

Figure 5 is a flow diagram of the strip function defined 
as block 234 in Figure 3; 

Figure 6 is a flow diagram of the sizing function defined 
25 as block 260 in Figure 3 ; 

Figure 7 is a flow diagram of the skin match function 
defined as block 2 64 in Figure 3; 

Figxire 8 is a flow diagram of the background gamma 
function defined as block 266 in Figure 3; 
30 Figure 9 is a flow diagram of the pixel enhancement 

function defined as block 268 in Figure 3; 

Figure 10 is a flow diagram of the position function 
defined as block 270 in Figure 3 ; 

Figure 11 is a flow diagram of the layering function 
3 5 defined as block 274 in Figure 3; 

Figures 12a and 12b are a flow diagram of the top-level 
gamma function defined as block 262 in Figure 3; 
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Figure 13 is a flow diagram of the fuzz function 
corresponding with block 530 in Figure 12; 

Figure 14 is a flow diagram of the blend function 
corresponding with block 540 in Figure 12; 
5 Figure 15 is a flow diagram of the change gamma function 

* corresp6nding with block 550 in Figure 12; 

Figure 16 is a flow diagram of the transparency /opacity 
function corresponding with block 560 in Figure 12; 

Figure 17 is a flow diagram of the gradient sharpening 
10 function defined as block 446* in Figure 9; 

Figure 18 is a block diagram illustrating the components 
of another embodiment of the imaging system; 

Figures 19a, 19b, and 19c are a top-level flow diagram of 
the imaging system of Figure 18; 
15 Figure 20 is a flow diagram of the mask function defined 

as block 103 0 in Figure 19b; 

Figure 21 is a flow diagram of the mask color structure 
fxinction defined as block 1110 in Figure 20; 

Figures 22a and 22b are diagrams of an exemplary mask 
20 color structure as used by the mask function of Figure 20; 

Figure 23 is a flow diagram of the Object 
Re-Color/Pattem option, defined at the brush options step 
1042 in Figure 19b; 

Figure 24a is an exemplary object image shown in 

25 isolation; 

Figure 24b is an exemplary image that includes a border 
which was drawn by the imaging system using the object of 
Figure 24a and using an Object Stanp option; and 

Figures 25a and 25b are exemplary images produced by the 
3 0 imaging system in performing the process steps shown in Figure 

19. 

Detailed Description of the Preferred Em bodiment 
Referring initially to Figure 1, a presently preferred 
35 embodiment of a digital image editing system of the present 

invention is generally indicated at 100. It is to be 
understood that the imaging system 100 can be used in 
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virtually any application where it is desirable to separate an 
object from a background in a digital image, and then combine 
the object with a different background to form a composite 
image . 

5 One such application is shown in Figure 1, which shows 

that the system 100 can be mounted by brackets 102 within an 
automated electrical imaging system 104 . The apparatus 104 
includes a video camera 106, such as a model VKC-360 camera 
available from Hitachi Corp., and which is electrically 

10 connected to the system 100. 

Still referring to Figure 1, a video monitor or display 
110, such as a Kodak model 1310 RGB/CGA touch screen display, 
is mounted in the apparatus 104 . The monitor 110 is 
electrically connected to the system 100 for displaying a 

15 video image, such as the image of a person 112 . As shown, the 

person 112 using the system (the user) in Figure 1 is sitting 
in front of a monochrome background 114, which can be any 
desired color. 

As shown in Figure 1, the apparatus 104 also includes a 
20 payment -receiving device 116, such as a model OB-A4 device 

made by Rowe. The presently preferred embodiment uses the 
device 116 to accept currency as payment . However, other 
embodiments may accept other forms of payment, such as credit 
cards and tokens. Further, the apparatus 104 includes a 
25 printer 118 which is filled with blank sheets of card stock 

120. Following deposit of an appropriate amount of currency 
in the device 116, or following other initiation, the image of 
the person 112 can be printed on one of the sheets 12 0 and 
dispensed into a printer bin 122 . In one presently preferred 
embodiment, the printer 118 is a model SV6510 color printer 
available from Kodak, 

Referring now to Figure 2, the system 100 is shown to 
include various electronic conponents. As indicated in Figure 
2, the system 100 includes a digital computer 130, preferably 
an IBM PC compatible having a 80386 microprocessor operating 
at 33 MHz and having eight Megabytes (Mb) of memory. As 
shown, the computer 130 is electrically connected to the video 
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camera 106 and payment adapter 116 for receiving input signals 
therefrom. Also, the computer 130 is electrically connected 
to a suitable graphics or video interface card 132, preferably 
a Targa®+ 16-32, available from Truevision having one Mb of 
5 video memory. Targa is a registered trademark of Truevision, 

Inc. The video memory on the graphics interface card 132 may 
store, at various times, a digital representation of part of 
the person 112 (Figure 1), a background scene, and 
instructions screen information for the user. The graphics 

10 interface card 132 in turn is electrically connected to the 

touch screen video monitor 110. The user of the system 100 
can respond to pron^ts given by the system by either touching 
or touching and then moving, (dragging) a location on the 
video monitor screen. A RS232 (serial format) digital 

15 connection from the video monitor 110 to the computer 13 0 then 

provides for the transfers of the user input to the system 
100. 

The computer 13 0 connects to the printer 118 via a 
Centronics compatible interface. The printer 118 is further 

20 connected to the video monitor 110 via an analog interface. 

Figure 2 also shows that if desired, an electronic storage 
device 134, such as a hard disk drive, can be connected to the 
computer 130. In the presently preferred embodiment, the hard 
disk 134 has a capacity of 120 Mb. 

25 Now referring to Figures 1, 3a, 3b and 3c, the operation 

of one presently preferred embodiment of the imaging system 
100 will be described. Figures 3a,b,c show the main or top- 
level control flow for the system 100. For the specific 
embodiment shown in Figure 1, execution begins at a start step 

3 0 202 of Figure 3a and proceeds to a step 204 wherein a set of 

background scenes is displayed on the video monitor 110 
(Figure 1) to attract the attention ("attract" mode) of a 
potential user of the system 100. Periodically the computer 
13 0 proceeds to a decision step 206 to check if someone has 

35 either touched the monitor screen 110 (Figure 1) at an 

appropriate location or inserted payment into the payment 
adapter 116 (Figure 1) , If there is no user input at step 
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206, the computer 130 loops back to step 204 to continue the 

"attract" mode. 

When there is user input at step 206, a decision step 208 
then checks whether the user inserted payment into the payment 
5 adapter 116, in which case the computer 130 proceeds to a 

decision step 210. At step 210, the payment is tested to 
determine if it is the proper amount and is genuine. In the 
preferred embodiment, the system 100 accepts only a five 
dollar bill. However, a configuration file (not shown) used 

10 by the computer 130 allows a technician to alter the type of 

payment that is accepted, e.g., five one dollar bills. At 
step 210, if the payment is rejected for any reason, the 
computer loops back to step 204 to begin again. 

If step 208 determines that the user payment has not been 

15 made, the computer 130 proceeds to step 212. At step 212, a 

voice file from a sound board in the computer 130 and voice 
driver activate to instruct the user to insert five dollars, 
sound boards, such as Sound Blaster® from Creative Labs or 
Thunder Board™ from Media Vision are readily available. Sound 

20 Blaster is a registered trademark of Creative Labs, Inc. 

Thunder Board is a trademark of Media Vision, Incorporated. 
A video clip, having instructions on how to make payment, is 
stored on the storage disk 134 (Figure 2) and is shown on the 
monitor screen 110 for about fifteen seconds. A check is made 

25 at a decision step 214 during the fifteen seconds to determine 

if payment has been made. If not, the computer loops back to 
step 204 to begin again. If payment is made at step 214, step 
210 tests for correct payment. If the payment is correct, the 
computer moves to step 216 wherein a RS232 acknowledgement is 

30 sent to the computer 130. The main control flow leaves Figure 

3a through the off -page connector A 218 and resumes again at 
step 220 in Figure 3b. 

At step 220, live video with a digital mask 140 (Figure 
4a) is shown on the video display 110. A voice file from the 

35 computer 130 activates to give the user instructions. Figure 

4a illustrates what the user 112 may see on the video display 
110. A box 142 delineates the edge of the digital mask 140 
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and defines a specific region where the head of the user 112 
needs to be. Voice instructions prompt the user 112 to sit 
back or to raise or lower the provided stool (not shown) if 
the head is not within the box 142. When the user's head fits 
5 properly within the box 142, further processing is not 

necessary to shrink the head size. The digital mask 140 is a 
monochrome hue that is configurable by a technician. 

Moving from step 220 to step 222 (Figiire 3b), the 
computer 130 activates the video camera 106 (Figure 2) and a 

10 video signal is captured representative of an object, e.g., 

the person or user 112, and a background against which the 
object was imaged, e.g., the continuous hue background 114 
(Figure 4a) . The computer 13 0 proceeds to a decision step 224 
to determine if this is the first picture or image captured. 

15 If so, the computer 130 loops back to step 22 0 and step 222 to 

capture a second pose of the user 112. When step 2 24 
determines that two images are captured, the computer 130 
proceeds to step 226 wherein both video images are digitized. 
Of course, the system could be modified to receive and process 

20 only one or more than two images. 

As shown in Figure 4a, only a portion 142 of the camera 
view is digitized for processing by the computer 13 0. 
Ordinarily, the signal from the camera 106 is an analog 
signal, and is digitized into a two-dimensional matrix of 

25 pixels or image 144 (shown in Figure 4b) by a video frame 

grabber on the Truevision Targa+ card 132. In the presently 
preferred embodiment, the matrix of pixels 144 is 512 X 486, 
which is a digital video rectangular standard, although other 
matrix sizes can be used such as 640 X 4 80 or higher. 

3 0 After both images have been digitized, ,the computer moves 

to step 228 wherein both images are displayed on the video 
display 110 (Figure 1) and a voice file instructs the user 112 
to pick one of the two images to be used for the rest of the 
process and to touch that image on the monitor screen 110 at 

35 step 230. The selected image 144 is stored in the video 

memory of the graphics interface card 132 (Figure 2) . After 
the user 112 makes the necessary selection at step 23 0, the 
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computer 130 proceeds to step 232 . At step 232, a set of 
different background scenes are displayed on the video display 
110. For example, the background scenes could be of sports 
team members in various poses with their equipment, either 
5 individually or in groups. In the preferred embodiment, the 

con^uter 130 displays a preselected set of background scenes, 
but in other embodiments, the user may be able to select a 
different set to choose among, e.g., another sports team. 
Other embodiments may allow the user to pick which person, 

10 from a group of people in a scene, that the user would like 

his picture to replace. This choice may be done by pointing 
to a person, pointing to the name or title of a person, or 
other ways of selecting the location for replacement. The 
user 112 is pron5>ted at step 232 to select one desired 

15 background scene by touching the appropriate scene on the 

video screen 110 (Figure 1) , 

While the user is deciding which background scene to 
select, the computer executes a function 234 to strip portions 
of the image selected by the user at step 230. Function 234 

20 will be explained in detail hereafter. When the user 112 

touches the monitor screen 110 (Figure 1) at step 236 to 
select the desired background scene, an interrupt is sent to 
the computer 130 to denote that the selection has been made. 
In the meantime, when the function 234 returns from execution, 

25 the computer 130 moves to step 238 wherein the results of 

function 234, i.e., the image of the user's head and neck 
along with the stripped background pixels, is written to the 
video memory on the graphics interface card 132 (Figure 2) . 
When the interrupt indicating the selection of the background 

30 scene from step 236 is received by the computer 130, the 

control flow 200 continues at step 240. 

At step 240, a personal computer keyboard and text entry 
screen along with instructions to the user 112 are displayed 
on the video monitor 110. A voice file from the computer 130- 

35 activates to give the user oral instructions- The user is 

prompted to enter text, such as the user's name or another 
name, to personalize the final composite image. While the 
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user 112 is thinking about the text to enter, the computer 130 
moves to step 242 where the background scene selected by the 
user at step 236 is sent to a frame store in a memory of the 
printer 118. As shown in Figure 4e, in the preferred 
5 embodiment, the backgro\ind scene sent to the printer 118 will 

be missing the head and neck of a preselected person 14 5 in 
the original conputer- stored scene. In other embodiments, the 
person in the original computer- stored background scene that 
is chosen to be replaced by the user (from multiple people in 

10 the scene) will be sent to the printer frame store without the 

head and neck. While the background is being written to the 
printer frame store at step 242, the user can press locations 
on the touch screen monitor 110 to choose characters composing 
the desired text at step 244. A location on the touch screen 

15 monitor 110 is provided to indicate completion of the text 

upon which the computer proceeds to a decision step 246. At 
step 246, a check is done to determine if a valid text entry 
has been made. If not, the computer 130 loops back to step 
242 where locations are provided on the touch screen monitor 

20 110 to correct the text entry. 

After a valid entry has been determined at step 24 6, the 
computer 130 moves to step 248 wherein the image of the face 
(Figure 4b) selected by the user 112 at step 230 is displayed 
on the video monitor 110 along with the original monochrome 

25 background present when the user's picture was taken. After 

con^letion of step 248, the main control flow 200 leaves 
Figure 3b through the off -page connector B 250 and resumes at 
step 252 in Figure 3c. 

At step 252, the computer 13 0 draws a horizontal 

30 reference line on the video display 110 and prompts the user 

112 to touch and drag (move while touching) the reference line 
over the pupils of the eyes. At step 254 the user 112 centers 
the horizontal line over the pupils and presses a button 
location on the touch screen monitor 110 to signal completion. 

35 of the step. Moving from step 254 to step 256, the computer 

13 0 draws a reference cross on the video display 110 and 
pron^ts the user 112 to touch and drag the reference cross to 
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the bottom of the chin and centered on the neck. At step 258 
the user 112 centers the reference cross on the middle of the 
neck and bottom of the chin, and then presses a button 
location on the touch screen monitor 110 to signal completion 
5 of step 258. The monitor 110 has a display similar to Figure 

4g but including the monochrome background 114 . 

Upon completion of step 258 in Figure 3c, the computer 
calls a function 260 to match the size of the user's face, as 
determined by steps 252 through 258 above, to the size of the 

10 face to be replaced of the previously selected person 145 

(from step 236) - The function 26 0 will be described below 
after the remainder of the main control flow 2 00 is described. 
After the function 260 returns, the computer calls a function 
262 to change various gamma values. 

15 The overall gamma includes many attributes: hue (H) , 

saturation (S) , lightness (L) , intensity (I) , contrast (C) , 
red (R) , green (G) , blue (B) , and combinations thereof such as 
HSL, HSI, HSC and RGB. The top-level gamma function 262 can 
change any combination of the gamma attributes by: pixel, area 

20 of the image or the entire image. For example, the user can 

change the hue, saturation and intensity of an area of the 
image. Other operations done by the top-level gamma function 
include: strip, fuzz, blend, transparency /opacity, and pixel 
enhancement. These functions, and the apparatus and method 

25 for accomplishing them, are disclosed hereafter in this 

document. For example, the user could choose to only enhance 
the pixels of a particular hue of blue, and blend the pixels 
of a certain saturation level . Any combination of gamma 
attributes and operations could be done. 

30 To more fully understand the application of the top-level 

gamma function 262, as well as other aspects of the invention, 
one should appreciate that the composite final image is 
handled by the computer 130 on a layer basis, where each pixel 
on a layer has X,Y Cartesian coordinates. Thirty two layers 

35 are used in the presently preferred embodiment, but in other 

embodiments the numbers of layers may be greater, with the 
maximum number only being limited by the size of the memory. 
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The final composite image is viewed with the layers stacked on 
top of each other. The layer number provides a Z coordinate 
with the original background scene having a Z coordinate of 
zero- Objects in the original backgroxmd scene can be chosen 
5 to have a higher or the same priority or similar and hence be 

assigned to a higher numbered layer and Z coordinate. Other 
objects, such as the image of the user 112, can be assigned a 
Z coordinate and be placed in front of or behind objects 
(depending on their Z coordinate) from the background scene 

10 which were previously moved from layer zero. 

As an example, consider a final composite image having 
four layers (Z=0 to 3) as shown in Figure 4h. If a pixel 150 
at a particular X,Y cartesian coordinate address of the top 
layer (Z coordinate = 3) has an attribute of transparent, then 

15 a pixel 151 at the same X,Y coordinate on the layer below 

(Z=2) will be seen if the attribute is opaque. However, if 
the pixel 151 of layer Z=2 is also transparent, then a pixel 
152 at the same X,Y coordinate on layer Z=l will be seen if it 
has an attribute of opaque and so on. For a pixel 153 on the 

20 background or Z=0 layer to be seen, all pixels on higher 

numbered layers for that X,Y coordinate address must all have 
an attribute of transparent. 

In further explanation of Figure 4h, several objects have 
been moved from the original background layer (Z=0) : two 

25 kneeling players 154 to the top layer (Z=3) and a hat 155 to 

the Z=l layer. The head of the user 112 is placed as an 
object 156 on the Z=:2 layer. The person 157 in the background 
scene whose head is to be replaced has the head area 158 set 
transparent. The final layered composite image 159 

30 illustrates the user's head 156 wearing the hat 155 on the 

body of the person 157. 

All the previously mentioned gamma attributes and 
operations can be done on a layer by layer basis. For 
example, the user can strip pixels of a particular hue of red 

35 from layer 1, 2 and 3, followed by shairpening pixels of a 

certain value of intensity on layers 0 and 1, and finishing 
with fuzzing all pixels with a desired saturation level on 
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layers l and 3. Details of the top-level gamma function 262 
will be given below. 

After a return from the function 262, the computer 130 
calls the function 264 to match the skin of the user 112 to 
the skin of the selected background person 14 5 (Figure 4e) . 
The function 2 62 will be described below. 

After the function 264 returns, the computer 130 calls 
the function 266 to match the gamma of the user image and the 
gamma of the selected background scene (Figure 4e) , as 
selected at step 236. The function 266 will be described 
below. 

In the presently preferred embodiment, after the function 
260 returns, the main control flow 200 will move to function 
268 and bypass functions 262, 264, and 266 (this bypass is not 
shown in Figure 3c) . However, it can be appreciated that the 
functions 262, 264, and 266, are included in other 
embodiments . 

After the function 266 returns, the computer 130 calls 
the function 268 for pixel enhancement. The function 268 will 
be described below. 

After the function 268 returns, the computer 130 calls a 
function 270 to position the object into the selected 
background scene (Figure 4e) . The fxmction 270 will be 
described below. 

Upon return from the function 270, the computer 130 will 
display, at step 272, a diversion screen, e.g., a team logo, 
on the video monitor 110 along with a message that the final 
image will be ready soon. While the diversion screen is 
displayed, the computer 130 proceeds to step 274. At step 
274, the computer 130 calls a function to layer the image of 
the user' s head and neck into the background scene chosen at 
step 236. The function 274 will be described below. 

After the function 274 returns, at step 276, the computer 
13 0 sends the user's head and neck block, as processed by the 
layering function, to the frame buffer of the printer 118 in 
overlay mode. In overlay mode, the block currently sent to 
the frame buffer overwrites the previous information in the 
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buffer starting at a defined location associated with the 
current block. The information previously in the frame buffer 
outside of the area for the block being overlaid remains 
unchanged. The final conposite image is built up at the 
5 printer memory. Previously, the background scene was sent to 

the memory of the printer 118 at step 242. 

Upon completion of step 276, the computer 130 moves to 
step 278 where the conposite image, which is almost complete, 
is sent from the printer memory to the video display 110. The 

10 computer 13 0 then proceeds to call the layering function 274 

again to layer the personalization text that the user entered 
at step 244. When the layering function 274 returns, the 
computer 130 proceeds to step 282 where the personalization 
text is sent in overlay mode to the frame buffer of the 

15 printer 118. At this point, the final composite is complete 

in the printer memory. The computer 130 moves to step 284 
where the personalization text is sent from the printer memory 
to the video display 110 to show the final composite image to 
the user 112 for a few seconds. As the final composite image 

20 is being displayed, the computer 130 moves to step 286 where 

a voice file from the computer 13 0 activates to thank the user 
112 for using the system 104. The computer 130 then signals 
the printer 118 at step 288 to print the final composite image 
on the printer card stock 120 and release the printed product 

25 into the printer bin 122. After the final composite image is 

printed at step 290, the main control flow 200 returns to step 
2 02 and begin the whole process again. 

Now referring to Figure 5, the function 234 (Figure 3b) 
for stripping, portions of an image will be described. The 

30 function 234 starts and moves to a step 310 wherein the 

computer 130 selects the upper left and lower right pixels of 
the video image 144 passed to the function as shown in Figure 
4b. It is to be understood, however, that the principles of 
the process described below, which is executed by the computer 

35 130, can be applied when selecting only a single pixel as a 

starting point or when selecting more than two pixels as 
simultaneous starting points . For example, as shown at Figure 
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4c, the computer 130 could execute the process described below 
starting at all four comers of the image 144 simultaneously. 
The process described below can be coded in any suitable 
computer language, e.g. C* 
5 Accordingly, for disclosure purposes, the process below 

will be described with reference to use of the upper left 
pixel of the digitized video image 144 as the starting point. 
It will be appreciated that the principles of the process will 
be substantially identical for other starting points, e.g., 
10 the lower right pixel of the video image 144, including 

instances wherein the computer 130 starts at two or more 
points simultaneously. The control of the parallel processing 
for multiple processing regions is known to those skilled in 
the technology. 

15 Step 312 indicates that the computer 13 0 compares the 

gamma of the upper left pixel (target pixel) with the gamma of 
the next pixel in the top row of the video image 144 (Figure 
4b) . As is used in the technology, the gamma of a particular 
pixel is a numeric value that refers to one or more of the 

20 particular data attributes which characterize the pixel. For 

example, some video image pixels have attributes that relate 
to the hue, intensity, lightness, saturation, and contrast of 
the portion of the image represented by the pixel. 
Accordingly, such pixels will have respective numeric "gammas" 

25 that represent each one of the above-listed attributes of the 

particular pixel . 

For purposes of the present invention, the gammas that 
are compared between pixels are the hue gammas of the pixels, 
although other gammas may be used. In the present embodiment, 

30 the hue gamma of each pixel is an integer value from zero (0) 

to two hundred fifty five (255), with the hue gamma value 
indicating the hue of the portion of the image represented by 
the pixel. In the event that the video image is in black-and- 
white, the hue gamma of a pixel will represent the shade of 

35 gray of the pixel, or a gray scale value. 

It will accordingly be appreciated that in the case of 
the continuous hue background 114 (Figure 4b) , adjacent 



wo 94/28490 PCT/US94/05756 

-25- 

background pixels will have substantially identical hue gamma 
values, with the particular value depending on the color of 
the background 114. For example, if the background was 
defined as 255, e.g., saturated blue, then the background 
5 pixels would typically not deviate more than 20 to 40. Thus, 

the conputer 13 0 assumes that the upper left pixel of the 
video image 144 is a background pixel, and uses this pixel as 
a standard pixel. The computer 130 coTi5>ares its hue gamma 
with the hue gamma of the immediately adjacent pixel (which 
10 can accordingly be considered the test pixel) that is in the 

same row as the target pixel, to determine whether the 
immediately adjacent pixel (i.e., test pixel) is also a 
background pixel. This step is represented at a decision step 
314 in Figure 5. 

15 More specifically, as indicated at decision step 314, the 

computer 130 con^ares the difference between the hue gammas of 
the upper left comer pixel (i.e., the target pixel) and the 
immediately adjacent pixel (i.e., the test pixel) with a 
predetermined difference. When the predetermined difference 

20 is greater than the difference between the hue gammas of the 

two pixels, indicating that the test pixel has approximately 
the same hue as the target pixel and is therefore flagged as 
a background pixel to be acted upon later, the computer 13 0 
proceeds from step 314 to a decision step 316 in Figure 5. At 

25 step 316, the computer 130 determines whether the test pixel 

is the last pixel in the row. If there are more pixels in the 
row, the computer 130 advances to step 318, wherein the 
computer 13 0 sets the old test pixel to be the new target 
pixel, and selects the next pixel in the row as the new test 

30 pixel. The computer 130 then continues to step 312 to 

determine the difference between the hue gamma of the target 
pixel and the hue gamma of the test pixel, as described above, 
and conpares this difference to the predetermined difference 
at decision step 314. 

3 5 On the other hand, when the computer 130 detearmines that 

the test pixel is the last pixel in the row at decision step 
316, the computer 130 proceeds to a decision step 320 that 
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determines whether the last row of the image 144 (Figure 4b) 
has been reached. Stated differently, at decision step 320, 
the computer 130 determines whether the row which had been 
\inder test is the bottom row of the digitized image, if the 
5 particular process is that portion of the processing that 

started at the upper left comer pixel of the digitized image. 
If, on the other hand, the particular process is that portion 
of the processing that started at the lower right comer pixel 
of the digitized image, the computer 13 0 determines whether 

10 the row that had been under test is the top row in the 

digitized- image- If not, the computer 130 proceeds to step 
3 22 wherein the computer 130 sets the target pixel to be the 
last pixel in the row and the test pixel to be the first pixel 
in the next immediately lower row. The computer 13 0 then 

15 loops back to step 312 to begin comparing the pixels of the 

next row. The computer 130 resumes the gamma test con^arison 
described above. Thus, the computer 130 reads and tests the 
pixels of the background component of the video image 144 
(Figure 4b) in a pixel -by-pixel, row-by-row sequence. 

20 When the coitputer 13 0 determines, at decision step 314, 

that the difference between the target pixel and the pixel 
under test exceeds the predetermined difference, indicating 
that the test pixel is not a background pixel and is therefore 
representative of an edge 14 6 of the image of the user 112 

25 (Figure 4c) which has been imaged against the monochrome 

backgroiind 114 (Figure 1), the computer 13 0 stores the 
location of the test pixel in memory. Stated differently, the 
computer 130 maps the test pixel as a portion of the edge 146. 
As indicated in Figure 5, the computer 130 then proceeds to 

30 step 320 and resumes processing as described above. 

The computer 130 follows the process described above, 
working simultaneously from the upper left and lower right 
hand corner pixels, until the computer 130 determines that no 
more pixels remain to be tested, as indicated by a positive 

35 test at decision step 320. At the point wherein no more 

pixels remain to be tested, the conputer 130 moves to step 
324, wherein the computer 130 "floods" the background (i.e.. 
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turns all pixels that were outside the edge 146 transparent) 
by setting the hue gamma of the background pixels to zero (0) . 
In other words, the computer 130 removes portions of the image 
144 (Figure 4b) which are represented by pixels that were 
5 designated as backgroxind pixels 114 in step 314. 

Alternatively, each background pixel could be turned 
transparent as soon as the computer 13 0 determines that the 
particular pixel is indeed a background pixel . The computer 
130 then moves to step 326 and returiis to the calling program. 
10 At this point in the process, the digitized video image 144 

appears as schematically shown at Figure 4d. 

Thus, the computer 130 effectively removes substantially 
only the portion of the video image 144 that is outside the 
edge 146 (i.e., the background con^jonent of the image), and 
15 leaves intact the entire portion of the image 144 that is 

inside the edge 146 (i,e., the object component of the image) . 
Consequently, portions of the image 144 inside the edge 146 
can have the identical hue as the background 114, without 
being removed from the image 144. Further, no particular, 
20 predetermined hue is required to be used as the background 

hue, in contrast to the chroma-key technology. Instead, any 
hue may be used as the background hue* 

From the disclosure above, it will be appreciated that 
the predetermined hue gamma difference is selected to be 
25 sufficiently large to avoid edge mapping due to a background 

test pixel that has a hue gamma which is marginally offset 
from the hue gamma of the target pixel. On the other hand, 
the predetermined hue gamma difference is selected to be 
sufficiently small to accurately detect the presence of an 
30 object edge pixel, even when the edge pixel has a hue gamma 

'value that is relatively close to the value of the background 
hue gamma. The precise value of the predetermined hue gamma 
difference can vary from application to application, depending 
upon lighting conditions, location, subject matter, etc. In 
35 one presently preferred embodiment, the predetermined hue 

gamma difference is 20. 
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Ref erring to Figure 6, the function 260 (Figure 3) for 
matching the size of the object to the size of the selected 
background object will be described. In the preferred 
embodiinent , the object is the head and neck of the user 112, 
5 and the selected background object is the head and neck of the 

background person 145 (Figure 4e) that will be replaced. 

The function 260 starts and moves to a step 342 wherein 
the computer 13 0 captures the face height of the user 112. 
Referring to Figure 4f , a portion of the background scene 
10 chosen by the user at step 236 (Figure 3b) is shown, including 

the backgroxind person 14 5 whose face and neck (shown by dashed 
lines) is to be replaced. A horizontal line 160 is centered 
over the pupils of the eyes at a coordinate yo. A cross 
symbol 162 is centered over the middle of the neck and at the 
15 bottom of the chin at a coordinate y^. A line 164 demarcates 

the edge of the clothing, e.g., a uniform, from which point 
downward the user's neck is assigned a lower priority layer 
than the uniform. Thus, the uniform may be seen to cover part 
of the user's neck in the final conposite image. The face 
20 height is the absolute difference between the two coordinates 

yo and yj. The face height information of the background 
person, or for each background person in background scenes 
where a choice of multiple people is given, has been 
previously calculated and stored in a file header for a file 
25 that contains the image of the background scene. The file 

header format for the presently preferred embodiment is 
encirypted and is unique to this embodiment. The header is 
encrypted by XORing with a pseudo-random function. The file 
header contains RGB information, location of the eyes, 
30 location of the chin (cross symbol), the face height, and so 

on to define the characteristics of the face or object in the 
file image. 

Referring to Figure 4g, an image of the user's face and 
neck is shown. A horizontal line 170 is centered over the 
35 pupils of the eyes at a coordinate y^,' . A cross symbol 172 is 

centered over the middle of the neck and at the bottom of the 
chin at a coordinate yi' . The user's face height is the 
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absolute difference between the two coordinates y©' and y^' . 
The user's face height information is stored in a file header 
for a file that contains the image of the user's head and 
neck. 

5 Now returning to Figure 6, after the completion of step 

342, the computer advances to step 344. At step 344, the 
computer 130 compares the user's face height captured at step 
342 to the face height of the background person at the 
location chosen by the user at step 236 (Figure 3b) - If, at 

10 a decision step 346, the face height of the user is less, 

e.g., the user 112 is a child, the computer moves to a step 
348. At step 348, the computer 130 will proportionately size 
up the image of the user's face and neck until the face height 
of the user equals the face height of the background person, 

15 and then returns at step 352. If, at a decision step 346, the 

face height of the user 112 is greater than that of the 
background person, the computer 130 moves to a step 350. At 
step 350, the computer 130 will proportionately size down or 
up the image of the user's face and neck, until the face 

20 height of the user equals the face height of the background 

person, and then returns at step 352. However if, at a 
decision step 346, the face height of the user 112 is equal to 
that of the background person, the computer 13 0 returns at 
step 352 to the calling main flow 200 (Figure 3) . 

25 Referring to Figure 7, the function 264 (Figure 3c) for 

matching the object or user person's skin to the skin of the 
selected backgroiind person 145 (Figure 4e) will be described. 
The function 264 starts and proceeds to a step 370 wherein an 
area of the background person, that is to be replaced in the 

30 con^josite image, is assigned to be measured. Moving to step 

372, the computer 130 measures three gamma attributes in the 
assicfned area: overall hue, overall saturation, and overall 
intensity. Next, at step 374, the computer 130 assigns an 
area of the user's skin to be measured. The computer 130 

35 knows where the eyes of the user 112 are on the image shown in 

Figure 4g because of the line 170, The computer assigns the 
area on the forehead of the image of the user 112 just above 
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the eyes. This area is then measured at step 376 for the 
same attributes: hue, saturation, and intensity. Proceeding 
to step 378, the computer 13 0 then determines the skin areas 
of the image of the user's face by using the attributes 
5 measured at step 376, which describe the user's skin, to be 

compared against upon calling the strip function 234. The 
function 234, in this call, will flag the skin pixels of the 
image of the user's face. The hair, eyebrows, eyes, lips, 
beard and/or mustache (if present) , and so forth will not be 

10 flagged. Then, upon return from the function 234, the 

computer 130 proceeds to step 382. At step 382, the flagged 
skin pixels identified by the function 234 are set to the 
values of the attributes measured at step 372 of the 
background person's skin, and then the function 264 returns at 

15 step 384 to the calling main flow 200 (Figure 3) . 

Referring to Figure 8, the function 266 (Figure 3c) for 
matching the object or user gamma to the gamma of the selected 
background scene (Figure 4e) will be described. The function 
266 can make it appear as though the object were imaged under 

20 the same lighting conditions as the background into which the 

object will be digitally layered. The function 266 starts and 
proceeds to a step 400 wherein the computer 130 determines the 
type of lighting used for the background scene (Figure 4e) . 
The lighting conditions are determined by measuring the gamma 

25 attributes of hue, saturation, and contrast at a software 

determined area of the backgrouind scene. Moving to step 402, 
the computer 130 measures the same gamma attributes as at step 
400, but for the user image 144 (Figure 4d) . 

Next, the computer 130 proceeds to step 4 04 wherein the 

30 computer 130 chooses which lighting condition is desired. In 

other words, the computer 130 at step 4 04 determines whether 
the hue gamma of the object will be changed to match the hue 
gamma of the background or whether the hue gamma of the 
background will be changed to match the hue gamma of the 

35 object. This determination by the computer 130 can be 

accomplished in response to an interactive command by the user 
.of the system 100, i.e., the user of the system 100 can decide 
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Whether he wishes to change the lighting condition of the 
background scene to match the lighting conditions under which 
he was imaged, or whether he wishes to change the apparent 
lighting conditions under which he was imaged to match the 
5 lighting conditions of the background scene. Furthermore, the 

apparent lighting conditions of the object and background 
scene can both be changed to match a third lighting condition 
if desired (this choice is not shown in Figure 8) • 

From step 404, the coitputer 13 0 proceeds to a decision 

10 step 4 06 wherein the computer 13 0 determines whether the 

object gamma is to be changed. If so, the computer 130 
proceeds to step 408 wherein the computer 130 sets the gamma 
attribute values for hue, saturation, and contrast of the 
object to equal the background scene gamma attribute values 

15 for hue, saturation, and contrast. Otherwise, the computer 

13 0 proceeds to step 410 wherein the computer 130 sets the 
gamma of the background scene to equal the gamma values of the 
object- t^n completion of either step 408 or step 410, the 
computer 130 proceeds to step 412 and returns to the main 

20 calling flow 200. 

Refeirring to Figure 9, the fxinction 268 for enhancing the 
pixels of an image will be described. The function 268 is 
called by either the main flow 200 (Figure 3c) or the top- 
level gamma function 262 (Figure 12) . The function 268 starts 

25 and proceeds to a step 420 wherein the computer 130 begins 

processing the image at a predetermined location, e.g., X,Y 
coordinates 0,0. Moving to step 422, the conqputer 130 
determines the hue value of each pixel on the current row to 
attempt to locate an edge of the object in that row. If an 

30 edge is found, as determined by a decision step 424, the 

computer 130 proceeds to a step 426. The edge is located if 
the computer 130 determines that the hue of the pixel has a 
major change con^ared to the previous pixel. In the preferred 
embodiment, such a change would be found if the value of hue- 

35 changed by an amount of 80 or more, where the full range of 

hue values runs from 0 to 255. Of course, the change 
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threshold could be any value within the above- indicated range, 
as selected for the particular application of the system. 

At step 426, the cornputer 130 selects the three pixels 
just outside the edge of the object and uses them as edge 
5 pixels in performing a blend operation. The blend operation 

is performed by the con^uter 13 0 in steps 428, 43 0, and 432 
wherein the aforementioned three pixels are blended to the 
backgrovmd layer directly below the current object layer. At 
step 428, for each of the three pixels, the computer 13 0 

10 determines the hue of the background pixel in the layer 

beneath the object pixel and then averages the hue value of 
each of the three background pixels with the corresponding 
object pixel at step 430. Moving to step 432, the computer 
130 sets the transparency or opacity of each of the three 

15 object layer edge pixels according to a blend factor that is 

found in the file header. The blend factor, having a value 
from zero (opaque) to 127 (transparent) , is used for dynamic 
variable transparency of the object, and is selected from 
within this range of values by the programmer using trial and 

20 error in view of what the results are desired to be. l^on 

completion of step 432, the computer moves to a step 434, 
wherein the computer 130 uses the edge found on the current 
row, as determined at step 422, and selects the two pixels 
inside the object edge for a fuzz operation. The fuzz 

25 operation is performed by the computer 130 in steps 436, 438, 

and 440 wherein the edge of the object is smoothed. At step 
436, for each of the two aforementioned pixels, the computer 
130 determines the hue of the three pixels immediately 
adjacent them on the same row. Then, as indicated at step 

3 0 438, the computer 130 determines the average hue value of the 

three-pixel interval. Moving to step 440, the computer 130 
sets the hue value of each of the two aforementioned pixels 
equal to the average determined in the previous step 438. 

Upon completion of step 440, the computer loops back to 

35 step 422 to attempt to locate another edge in the current row. 

If another edge is found, as determined at the decision step 
424, the previously described process is performed. However, 
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if the end of the current row is reached without locating an 
edge, the computer 130 moves to a decision step 442 to 
determine whether the last row of the image has just been 
processed. If not, the conqputer 13 0 proceeds to step 444 and 
5 selects the next row for processing and then loops back to 

step 422 to locate an edge. However, if the last row has been 
processed, as determined by decision step 442, the conputer 
calls a gradient sharpening function 446 to enhance the object 
edges . 

10 The file header has predetermined values for the amount 

of sharpening to be done. For exattqple, the image could have 
no sharpening done on the edges but full sharpening done at 
the center, 100% with a linear scale in between the center and 
borders. Other percentages can be used as a sharpening 

15 factor. For example, the image can have 30% sharpening at 

left edge, 2 0% at center and 80% at right edge with a linear 
scale in between the center and borders. The function 446 
will be described hereinbelow. After the function 446 has 
returned back to the function 268, the computer 13 0 exits the 

20 pixel enhancement function 268 at a step 448. 

The details of the process carried out by the computer 13 0 
at function 270 of Figure 3c are more fully understood in 
reference to Figure 10. The function 270 for positioning the 
object or user 112 into the selected background scene (Figure 

25 4e) will be described. The function 270 starts and proceeds 

to a step 460 wherein the computer 13 0 gets the address of the 
cross symbol 172 (Figure 4g) on the user image. This symbol 
is at the middle of the neck and bottom of the chin on the 
user image as placed by the user 112 at step 258 of Figure 3c. 

3 0 The address of the cross symbol 172 is retrieved from the file 

header of the user image file. The computer 130 then proceeds 
to step 462 and retrieves the address of the cross symbol 162 
(Figure 4f) for the selected person 14 5 of the background 
scene. This address is retrieved from the file header of the 

35 background scene image file. The computer then sets the 

address of the user's cross symbol 172 equivalent to the 
address of the cross symbol 162 of the selected person 145 in 
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the background scene. Therefore, when the composite image is 
layered together, the image of the user 112 will be at the 
correct location. The computer 13 0 returns at step 464 to the 
calling main flow 200 (Figure 3) . 
5 The details of the process carried out by the computer 

130 at function 274 of Figure 3c are more fully understood in 
reference to Figure 11. The function 274 for layering an 
object or personalization text into the selected background 
scene (Figure 4e) will be described. The function 274 is 

10 called twice by the main control flow 200. The first call in 

the preferred embodiment is to layer the image of the user's 
head and neck, such that user image will result being slid 
down, for example, a jersey or uniform, in the final composite 
image. The jersey has a higher priority than the user image, 

15 such that the user' s neck and head become visible at the top 

of the jersey. Both the jersey and the user image have a 
higher priority than the background scene. 

The second call in the preferred embodiment is to layer 
the personalization text which the user has entered at step 

20 244 (Figure 3b) - The text has the highest priority and, 

therefore, will always be visible in the final composite 
image . 

The function 274 starts and proceeds to a decision step 
4 80 wherein a determination is made whether this routine is a 

25 fiinction call, as in the preferred embodiment, or is a user 

call, for an alternate embodiment. In the preferred 
embodiment, the cotrqputer 130 proceeds to step 482 wherein the 
item, e.g., text or an object, or the area of an image to be 
layered is identified in the argument for the function call. 

3 0 The computer 130 then accesses the information corresponding 

to the item or area and proceeds to step 488. If however, at 
step 480, the call to this routine is a user call, the 
con^uter 130 proceeds to a step 484. At step 484, an area or 
object in the background scene can be defined by the user 112 

3 5 to be moved to another layer. At step 486, the selected area 

or object is stored to memory along with information for name. 
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layer, position within the layer, and alpha-bit flag used to 
set transparency bits, 

t^on completion of either step 486 or 482, the name, 
layer, position within the layer, and alpha-bit flag, all 
5 corresponding to the item or area, are written to a linked 

list by the computer 130 at step 488. The alpha-bit flag 
denotes transparency or opacity of the item or area. The 
position is stored in X,Y coordinates. After complet ion 

of step 4 88, the computer 13 0 moves to a decision step 490 and 

10 checks to see if an option is invoked, in an alternate 

embodiment, to stitch, or link together, multiple objects on 
the same layer. If so, at option step 492, the computer 
stitches the objects desired on the same layer together into 
a layer collection. After completion of option step 4 92 or 

15 if the decision step 490 is false, the conputer 130 moves to 

step 494. At step 494, the con^uter 130 invokes the linked 
list and looks for the transparency alpha-bit flags. At step 
496, for the item pointed to by the linked list, the computer 
130 displays the object or area defined by name, layer, and 

20 position within the layer on the. monitor 110 (Figure 2) . 

In cin alternate embodiment, a check is made at an option 
decision step 498 to determine if the user 112 (Figure 1) 
desires to reset the transparency alpha-bit flag. If so, the 
computer moves to option step 500 wherein the transparency 

25 flag is reset to opaque by use of a pointing tool, e.g. a 

mouse, or by area definition using X,y cartesian coordinates. 
Upon completion of step option 5 00 or if the decision step 4 98 
is false, the computer 130 moves to an option decision step 
502. In an alternate embodiment, step 502 determines if the 

3 0 user 112 wants to stitch an object or objects on one layer 

together with the background. If so, at option step 504, the 
computer 130 stitches the objects on one layer together with 
the background into one collection aind then the objects are 
removed from the linked list- The collection may include 

35 objects, an area, backgroxinds , text, and so forth. In the 

presently preferred embodiment, a file format for storing the 
background, or the collection as a new background, has a file 
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extension ".spo". At the completion of option step 504 or if 
decision step 502 is false, the computer 130 returns at step 
506 to the calling main flow 200 (Figure 3) . 

Referring to Figure 12a and 12b, the top-level gamma 
5 fxanction 262 (Figure 3c) will now be described. Although the 

preferred embodiment disclosed herein does not include the 
top-level gamma function 262, an alternate embodiment may 
include this function. The function 262 may be utilized in 
two ways: as a function call, wherein attributes to be changed 

10 are preselected, or as a user call, wherein attributes to be 

changed are selected by the user 112. When function 262 is 
accessed by a user call, the system 100 operates as a highly 
interactive and very powerful image editing tool. 

The function 262 starts and moves to a decision step 514 

15 to determine if the routine 262 is called as function. If so, 

the coit^uter 130 proceeds to a step 516 to access preset gamma 
attributes, layer, and object or area selections, e.g. , all 
pixels on layer one with a hue of blue at a value range of 10 
to 75. The computer 13 0 then moves from step 516 and 

20 proceeds through the off -page connector C 580 and resumes 

again at step 582 on Figure 12b. However, if the routine 262 
is a user call as determined at step 514, the computer 130 
proceeds to a decision step 520 wherein the con^uter 130 
checks to determine if a strip operation is to be done. The 

25 strip operation in this function may be done on an object or 

on the background scene. If so, the computer 13 0 proceeds to 
step 522 wherein the user 112 selects the gamma attributes, 
with corresponding values, and/or a layer and an object or an 
area on the layer to be changed. At the completion of step 

3 0 522, the conputer 130 proceeds to step 524 and enters the 

selections from step 522 into a matrix. At a decision step 
526, the computer 130 checks to see whether the user 112 wants 
to undo the operation selected at step 520. If so, at step 
528, the computer 130 will negate the operation selected at 

35 step 520. 

The x-cixis of the matrix mentioned above includes the 
gamma attributes: hue (H) , saturation (S) , lightness (L) , 
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intensity (I) , contrast (C) , red (R) , green (G) , blue (B) , and 
the combinations HSL, HSI, HSC and RGB. The y-axis of the 
matrix includes the operations done by the top-level gamma 
function: strip, fuzz, blend, change gamma, 

5 transparency/ opacity, and pixel enhancement. Associated with 

the matrix is a z-axis that corresponds to the layer number of 
a composite image. In the presently preferred embodiment, the 
z-axis utilizes 32 layers. The matrix permits one or multiple 
operations on one or multiple gamma attributes to be perf birmed 
10 on one or multiple layers of the composite image in any 

combinat ion . 

After completion of step 528, or if decision step 526 
result is false, or if the decision step 520 result is false, 
i.e., strip was not selected, the computer 130 proceeds to a 

15 decision step 530 wherein a check is made to determine if a 

fuzz operation is to be done. The fuzz operation may be done 
on an object or on the background scene. If so, the computer 
moves to a set of steps essentially the same as steps 522 
through 528 as described above and then proceeds to a decision 

20 step 540. If the check to determine if the operation is to be 

done at step 530 (and steps 540, 550, and 560 below) is false, 
the computer 13 0 continues to the next decision step to test 
for the next possible operation. 

At step 54 0, the computer 130 checks to determine if a 

25 blend operation is to be done. The blend operation in this 

function may be done on the background scene or on an object 
on the adjacent layer. If so, the computer moves to a set of 
steps essentially the same as steps 522 through 528 as 
described above and then proceeds to a decision step 550. 

30 At step 550, the computer 130 checks to determine if. a 

gamma change operation is to be done. The gamma operation in 
this function may be done on an object and/or the background 
scene. If so, the computer moves to a set of steps 
essentially the same as steps 522 through 528 as described 

35 above and then proceeds to a decision step 560. 

At step 560, the computer 13 0 checks to deterroine if a 
transparency/ opacity operation is to be done. The 
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transparency /opacity operation in this function may be done on 
the background scene or on an object on the adjacent layer. 
If so, the computer moves to a set of steps essentially the 
same as steps 522 through 528 as described above and then 
5 proceeds to a decision step 570 , 

At step 570, the computer 13 0 checks to determine if a 
pixel enhancement operation is to be done. The pixel 
enhancement operation in this function may be done on an 
object and/or the background scene. If so, the computer moves 
10 to a set of steps essentially the same as steps 522 through 

528 as described above and then proceeds to step 582 via the 
off -page connector C 580 to Figure 12b. If the pixel 
enhancement operation was not selected at step 570, the 
computer 130 also proceeds to step 582 via the off -page 
15 connector C 580. 

The next three steps (582, 584, 586) of function 262 are 
recursively performed by the computer 13 0 for each preset 
(from step 516) or selected operation (from steps 520 to 570) 
in preparation for step 590. At step 582, the con^uter 130 
20 performs a layer limit address test. In the presently 

preferred embodiment there are 32 layers. The computer 130 
tests to see if it is at the top layer (Z=31) or at the bottom 
layer (Z=0) . Moving to step 584, the computer 13 0 prioritizes 
the layers and operations . The first operation that is 
25 prioritized is transparency/opacity . Layer zero (Z=0) is the 

original background scene. As an example, for an area on 
layer zero to be opaque, nothing in the same area on the 31 
layers above layer zero can be opaque, so therefore that area 
would be set transparent in each of the 31 layers. As another 
3 0 example, if layer seven (2=7) is made semi-transparent, then 

all the layers above layer seven (Z=8 to 31) must be 
transparent, and the layers below (Z=0 to 6) layer seven are 
adjusted in transparency/opacity to account for the semi- 
transparency of layer seven. In another example, the top 
35 layer (Z=31) is made opaque, so nothing needs to be changed on 

the other 31 layers. The other preset or selected operations 
are then also prioritized. At step 586, the computer 130 
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looks for the layer that is affected for each preset or 
selected operation. There could be six different layers 
corresponding to the six operations from steps 520 to 570 in 
the priority list. The steps 582 through 586 are recursively 
5 performed until the layer limit address test is satisfied for 

each operation. 

At step 590, the computer 130 goes through a process to 
schedule the execution of the operations selected in steps 520 
through 570, or from step 516, as described above, on a layer 

10 by layer basis. At a decision step 592, the con^uter 130 

checks to see if the operation scheduled for execution is the 
strip operation. If so, at step 594, the computer 130 
performs the strip on a layer (chosen for execution by step 
590) using the method that starts at the four comers of the 

15 layer and advances through the rows, as previously described 

in conjunction with fuinction 234. If the test at step 592 is 
false (not strip), one operation, e.g., fuzz, as scheduled by 
the computer 130, is executed on a layer (chosen for execution 
by step 590) using the row by row method of execution- After 

20 all the rows of a layer have been processed at step 596, the 

computer 13 0 advances to a decision step 598 and checks 
whether more layers are to be processed. If so, the computer 
130 loops back to step 590 to schedule the next selected layer 
with the same operation as previously executed, e.g., fuzz. 

25 One operation is done on all desired layers before moving to 

a decision step 600 wherein a check is made to determine if 
additional operations are to done. If so, the computer 130 
loops back to step 590 wherein the next selected new 
operation, e.g., blend, is scheduled to be executed. If all 

30 operations are complete at step 600, the conputer returns at 

step 602 to the calling main flow 200 (Figure 3) . 

The top-level gamma function 262 can be used to create 
interesting composite images. As an exan5>le, the user desires 
the image of his head to replace the image of the head of a 

35 hockey player on a hockey playing card. The hockey player has 

a big scar on his face which the user wants to have on the 
image of his face in the resultant composite image. Using the 
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top-level gatnma function 262, the image of the scar can be 
isolated and made opaque. The rest of the image of the hockey 
player's head is made transparent. The image of the user's 
head is made opaque except for the area where the scar is, 
5 which is made transparent. In the resultant composite image 

playing card, the scar appears on the image of the user's head 
which is on the image of the hockey player's body. 

Four functions execute the operations as selected by 
steps 530, 540, 550 or 560 of the top-level gamma ftmction 262 

10 (Figure 12) • The actual scheduling and call for execution of 

these functions is carried out by steps 590 and 596, However, 
for sake of reference, the operation selected by step 530 is 
referred to as a fuzz fionction 530', the operation selected by 
step 540 is referred to as a blend function 540', the 

15 operation selected by step 550 is referred to as a change 

gamma function 550' , and the operation selected by step 560 is 
referred to as a transparency /opacity function 560' . These 
four functions will be described hereinbelow. The operation 
selected by step 520 is performed by the strip function 234 as 

20 described in conjunction with Figure 5. The operation 

selected by step 570 is performed by the pixel enhancement 
function 268 as described in conjunction with Figure 9. 

After the background con^jonent 114 of the digital image 
144 (Figure 4b) has effectively been removed, the component 

25 representing the image of the user 112 remains (Figure 4d) , 

and has a relatively « fuzzy" edge, characterized by a pixel 
width of about one or two pixels. Accordingly, to produce a 
digital image having a smooth edge the digital computer 130 
executes the fuzz function 53 0' as shown in Figure 13. The 

30 fuzz function 530' may be called by either the pixel 

enhancement function 268 (Figure 9) or the top-level gamma 
function 262 (Figure 12) . The function starts at a step 530' 
and proceeds to a step 680 wherein the computer 130 selects 
one of the edge pixels (i.e., a "test" pixel) by using a 

35 software determined address (e.g., by selecting the uppermost 

left pixel) and determines its hue at step 682. Next, as 
indicated at step 684, the computer 130 selects the edge 



wo 94/28490 PCT/US94/05756 

-41- 
pixels which are immediately adjacent the test pixel for a 
three -pixel interval on one row and determines the hue of 
these pixels . 

Then, as indicated at step 686, the computer 130 
5 determines the average hue gamma value of the three pixels . 

As indicated at step 688, the computer 13 0 then sets the hue 
gamma value of the test pixel to be equal to the average value 
calculated in step 686. The computer 130 proceeds to a 
decision step 690, wherein the computer 130 determines whether 

10 the test pixel is the last edge pixel to be processed in the 

fuzz function 530'. If not, the computer 130 proceeds to step 
692, wherein the computer 13 0 selects one of the edge pixels 
that is immediately adjacent the test pixel, designates this 
adjacent pixel as the new test pixel, and loops back to step 

15 682. If, at step 690, the computer 130 determined that the 

test pixel was the last edge pixel, the computer 130 exits the 
fuzz function 530' at a step 694. 

The computer 13 0 blends the surrounding background to the 
edges of an object or an area on the layer adjacent to 

20 background scene layer in function 540' . The blend function 

540' is more fully understood by reference to Figure 14. The 
blend function 540' may be called by either the pixel 
enhancement function 268 (Figure 9) or the top-level gamma 
function 262 (Figure 12) - The computer 130 starts at step 

25 540' and proceeds to step 700 wherein the computer 130 selects 

an object edge pixel as the test pixel. The selection of the 
object edge pixel is made at a software determined address, 
e.g., the left edge pixel of the first row of the object. 
Moving to step 702, the computer 130 determines the hue gamma 

30 value of the test pixel. From step 702, the computer 130 

proceeds to step 704 and determines the hue of the background 
pixel that is on the layer immediately below the selected test 
pixel in the same row as the test pixel. Then the computer 
130 proceeds to step 706 wherein the computer 130 determines 

35 the average hue of the two pixels. From step 706, the 

computer 130 proceeds to step 708, wherein the computer 130 
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sets the transparency/opacity of the test pixel according to 
the blend factor in the calling function. 

From step 708, the computer 130 proceeds to a decision 
step 710, wherein the computer 130 determines whether there 
5 are other edges in the current row; that is, whether the test 

pixel is the last edge pixel to be processed in the current 
row. If there is at least one additional edge in the current 
row, the computer moves to step 712 and selects the next edge 
pixel. This pixel is designated as the new test pixel, and 

10 then the computer 130 loops back to step 702. However, if 

there are no more edge pixels in the current row as determined 
at step 710, the computer 130 advances to a decision step 714 
wherein a check is made to determine if the last row of the 
object has been processed. If not, the computer 130 proceeds 

15 to step 716, wherein the computer 130 selects the next row and 

the first edge pixel in that row. This pixel is designated as 
the new test pixel, and then the con^uter loops back to step 
702. If, at step 714, the con^uter 130 determined that the 
last row of the object has just been processed, the computer 

20 130 exits the blend function 540' at a step 718. 

Referring to Figure 15, the function 550' for changing 
the gamma attributes of an image will be described. The 
fvmction 550' is called by the top-level gamma function 262 
(Figure 12) . A list of the gamma attributes that may be 

25 changed was disclosed in conjunction with Figure 12. The file 

header may contain information about the area or object for 
which a gamma attribute or multiple attributes is to be 
changed. Depending on the complexity of the background, the 
header may contain additional information necessary to 

30 properly achieve photo- realistic events. The chcuige gamma 

function allows the user to change one or more gamma 
attributes on a whole layer or on an area or object on the 
layer. For example, the saturation attribute can be pumped up 
from a value of 40 to a value of 80 on the whole layer. 

35 The function 550' starts and proceeds to a step 73 0 

wherein the computer 130 begins processing at a software 
determined location of the image. This location may be at the 
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X,Y coordinates of the top left comer of an object or area in 
one embodiment or it may be the entire layer. Moving to step 
732, the computer 130 changes the pixel value of the selected 
attribute, e.g., hue. The computer 13 0 then moves to a 
5 decision step 734 to determine if additional preselected 

pixels (of the object, area, or layer) in the current row are 
to have an attribute changed. If so, the computer 130 moves 
a step 736 and advances to the next pixel in the current row. 
The computer 130 then loops back to step 732. If there are 

10 no additional preselected pixels in the current row as 

determined by step 734, the computer 130 moves to a decision 
step 73 8 to determine if the last row of the selected area has 
been processed. If not, the con^uter 130 moves to step 740, 
advances to the first pixel on the next row of the selected 

15 area, and then loops back to step 732. However, if the last 

row of the selected area, object, or layer has been processed, 
the conputer moves to a decision step 742 wherein a 
determination is made whether there are any more attributes to 
change. If so, the computer 130 moves to step 744, gets the 

20 next gamma attribute for change, and then loops back to step 

730. If there are no further attributes to change, as 
determined by step 742, the computer 130 exits the change 
gamma function 550' at a step 746. 

Referring to Figure 16, the transparency/opacity function 

25 560' will be described. The fiinction 560' is called by the 

top-level gamma function 262 (Figure 12) . The fiinction 560' 
starts and proceeds to a decision step 770 wherein the 
con^juter 130 determines whether the function is to be 
performed on an object or on a background scene. If it is 

30 determined at step 770 that an object is to be processed, the 

computer 130 moves to step 772 to begin processing at the 
first row of the object as defined in the file header. The 
computer 130 selects an object edge pixel on the current row 
as the test pixel. The selection of the object edge pixel is 

35 made at a software determined address, e.g., the left edge 

pixel of the first row of the object. Moving to step 774, the 
computer 130 determines the transparency /opacity (T/O) value 
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of the test pixel. From step 774, the computer 130 proceeds 
to step 776 and determines the T/0 of the background pixel 
that is on the layer immediately below the selected test pixel 
in the same row as the test pixel. Then the computer 130 
5 proceeds to step 778 wherein the computer 130 determines the 

average T/O of the two pixels. From step 778, the computer 
130 proceeds to step 780, wherein the computer 13 0 sets the 
transparency/opacity of the test pixel according to the blend 
factor in the calling fionction. 

10 From step 780, the computer 130 proceeds to a decision 

step 782, wherein the con^uter 130 determines whether there 
are other edges in the current row; that is, whether the test 
pixel is the last edge pixel to be processed in the current 
row. If there is at least one additional edge in the current 

15 row, the computer moves to step 784 and selects the next edge 

pixel. This pixel is designated as the new test pixel, and 
then the con^juter 130 loops back to step 774. However, if 
there are no more edge pixels in the current row as determined 
at step 782, the computer 130 advances to a decision step 786 

20 wherein a check is made to determine if the last row of the 

object has been processed. If not, the computer 130 proceeds 
to step 788, wherein the computer 130 selects the next row and 
the first edge pixel in that row. This pixel is designated as 
the new test pixel, and then the computer loops back to step 

25 774. If, at step 786, the computer 130 determines that the 

last row of the object has just been processed, the computer 
130 moves to a step 790. At step 790, the computer 130 
initializes further processing by accessing the first pixel of 
the first row of the area made transparent by use of the strip 

30 function 234 (Figure 5) . 

If the decision step 770 determination is that the 
function 560' is processing the background scene, the computer 
130 moves to a step 800 and accesses the first pixel of the 
first row of the image as the test pixel . After completion of 

35 step 800, the computer 130 moves to a step 802 wherein the 

file header is utilized to determine which of hue, red, green, 
blue (RGB), and intensity tests are to be run. Any 
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cotnbination or all five tests can be run. The test is to 
determine the hue, red, green, blue, or intensity value of the 
pixel. For each test, a range is kept in the file header 
along with information that denotes whether the test on the 
5 test pixel is for values that are inside the range or for 

outside the range. The range and the inside/outside 
information can be different for each of the five tests. As 
an exan^le, a predetermined limit on hue values may be from 
zero to 255 and the test may look for pixels that fall outside 

10 the hue value range of 50 to 100 . These pixels will have 

their alpha-bit flag set. Thus, of the steps 804, 808, 812, 
816, and 820, only those tests selected to run, according to 
the file header, will have their respective ranges checked. 
So, for the example just above, the computer 130 moves to a 

15 decision step 804 and determines whether the hue value of the 

test pixel is outside the range specified in the header, e.g. 
50 to 100. If so, the computer 130 sets the alpha-bit flag to 
transparent at step 806. However, if the pixel value does not 
match the range specification in the file header for hue, as 

20 determined by step 804, the computer 130 moves to the next 

decision block as selected in the file header, if any, among 
steps 808, 812, 816, and 820* A set of steps similar to 
steps 804 and 806 above is executed for each test selected in 
the file header. 

25 At the conclusion of the one to five tests above, the 

conqputer 13 0 moves to a decision step 824 and determines if 
there is at least one additional pixel in the current row to 
be tested. If so, the computer 13 0 advances to step 826, 
accesses the next pixel in the current row, and loops back to 

30 step 802 to repeated the above -described process. If the last 

pixel on the current row has been processed, as deteinnined by 
step 824, the conputer 130 moves to a decision step 828 to 
determine if the last row has just been processed. If not, 
the cotiputer 130 moves to step 830, advances to the next row 

35 of the image, and loops back to step 802. If the last row has 

been processed, as determined by step 828, the computer 130 
exits the transparency/opacity function 560' at step 832. 
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After completion of step 790, the computer 13 0 moves to 
step 802. Steps 802 to 832 are then similar to that described 
above except that for steps 806, 810, 814, 818, and 822, the 
alpha-bit flag is set opaque rather than transparent, because 
5 the area to be processed is already transparent (when step 802 

is reached via step 790) • As an example, setting the alpha- 
bits opaque allows the user to make a drop shadow in the shape 
of the user's head, which is known as shadow preservation. 

Referring to Figure 16, the gradient sharpening function 

10 446 will be described. The fxinction 446 is called by the 

pixel enhancement function 268 (Figure 9) . As an example, 
the function 446 may be used to locate the eyes on an image of 
a face so that the eyes could be enhanced while the skin is 
left unchanged. The function 446 starts and the computer 130 

15 proceeds to a step 850 and accesses the first pixel of the 

first row as the test pixel of the image that is passed to 
function 446 from function 268. Moving to step 852, the 
con^uter 130 measures the hue of the test pixel and advances 
to a decision step 854. At step 854, the conqputer determines 

20 whether the hue value falls outside a range specified by the 

file header, e.g., HO to 150. If so, e.g., the value is less 
than 110 or greater than 150, the computer 130 moves to step 
856 wherein the alpha-bit flag is set for that pixel. At the 
completion of step 854 or if the hue value does fall in the 

25 range, as determined at step 854, the computer 130 moves to a 

decision step 858. At step 858, the computer determines if 
there is at least one additional pixel left in the current 
row. If so, the computer 130 moves to step 860, accesses the 
next pixel in the current row and loops back to step 852. 

30 However, if the computer 130 determines, at step 858, that the 

last pixel in the current row has been processed, a decision 
step 862 determines whether the last row has just been 
processed. If not, the computer 130 moves to step 864, 
accesses the next row, and loops back to step 852 to process 

35 the new row. 

If the determination is made, at step 862, that the last 
row has been processed, the computer 13 0 moves to a step 870 
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to look for the first row having a pixel with the alpha-bit 
flag set. Moving to step 872, the computer 130 does edge 
enhancement on the pixel. The computer 130 performs one, two 
or three operations on the pixel, depending on the hue value 
5 of the pixel and a set of ranges for the hue value and 

associated operations which are kept in the file header. The 
three operations are: saturate the pixel, shift the hue (make 
the blacks blacker and the whites whiter) , and shift the 
intensity. A set of hue value ranges are set up in the file 

10 header such that, for example, if the measured hue value 

exceeds the file header value by more than 40, the saturation 
operation is done; if the measured value exceeds the header 
value by more than 70, the shift intensity operation is done; 
and if the measured value exceeds the file header value by 

15 more than 100, the saturation and the shift hue operations are 

done. The hue value ranges are backgroxind specific, and are 
set by the programmer to achieve the desired results based on 
that background. For example, if the skin color of the 
background object is green, the range will be selected so that 

20 the skin color of the object being placed in the background 

will also match this green color. 

Moving to a decision step 874, the con^uter 130 
determines whether there is at least one more pixel in the 
current row that has the alpha-bit flag set. If so, the 

25 computer 130 proceeds to step 876, accesses the next pixel in 

the current row having the alpha-bit set, and loops back to 
step 872- However, if decision step 874 is false, the 
con5>uter 13 0 moves to a decision step 878 to determine if the 
last row of the image has just been processed. If not, the 

30 computer 130 moves to a step 880, advances to the next row 

having a pixel with the alpha-bit set, and loops back to step 
872. If the last row has been processed, as determined by 
step 878, the gradient sharpening function 446 returns at step 
882 to the pixel enhancement function 268. 

35 The software described herein for the image editing 

system 100 is written in the "C" language and was translated 
from source code to machine -readable object code using a Photo 
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Novelty Programming Language (PNPL) compiler developed by 
ImageWare Software, Inc., formerly Practically Perfect 
Productions, Inc. of San Diego, CA. Nonetheless, one skilled 
in the technology will recognize that the steps in the 
5 accompanying flow diagrams can be implemented by using a 

number of different compilers and/or programming languages. 



Referring to Figure 18, another embodiment of an image 
editing system of the present invention is generally indicated 
10 at 900, The imaging system 900 can be used in virtually any 

application where it is desirable to separate an object from 
a background in a digital or video image, and then to combine 
the object with a different backgroxmd to form a new image. 
One such application is shown in Figure 18, which shows the 
15 system 900 integrated with a group of computer peripherals. 

The imaging system 900 includes an advanced photo- 
realistic image-editing and manipulation program operating in 
the popular Windows™ environment on a computer. Windows is 
available from Microsoft Corporation. Although this 

20 embodiment is described using the Windows environment on a 

personal computer, other embodiments may use a different 
environment or a different con^uter or both. 

With the imaging system 900, images are manipulated 
faster and more easily than ever before. Most image-editing 
25 programs offer the ability to manipulate areas of images, but 

with the imaging system 900, one or more objects in each image 
can also be mauiipulated. These objects are easily moved 
around, enlarged, reduced, or rotated, and even layered over 
other objects. Objects can be saved for later use with the 
3 0 same or different background files. This ability to select 

and work with as many objects as desired, set in the familiar 
and easy-to-use Microsoft Windows environment, provides more 
flexibility, ease, and speed in image editing. 

In addition to the versatility of multiple objects, the 
35 imaging system 900 offers many other features in creating 

professional graphic images. With the variety of selection 
tools, painting tools, and special effects available in the 
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imaging system 90 0, images are enhanced with functions as 
simple as sharpening or as advanced as gradually blending one 
image into another image beneath it. Complete color editing 
is available, and text is easily added to images. The term 
5 color herein is used to describe hue and shades of hue within 

a particular color, but also includes hues of gray including 
the absolutes of black and white. Font style and size of text 
is easily adjusted, and can be rotated just like an object. 
Despite the advance capabilities of the imaging system 900, 

10 there is no need to worry aJDOUt mistakes, which are easily 

corrected with an "undo" function. 

Referring again to Figure 18, the system 900 includes a 
computer 902 having a minimum of an Intel 80386SX or similar 
microprocessor that will run Microsoft Windows version 3 . 0 or 

15 higher in enhanced mode. A minimum of four megabytes (Mb) of 

RAM memory (not shown) plus a permanent swap file is required, 
and preferably eight Mb of memoary or more is desired for 
• professional level manipulations. In general, the more memory 
the computer 902 has, the faster the system 900 performs. 

20 The system 900 includes a hard disk drive 904 connected 

to the computer 902. The drive 904 has at least four 
megabytes of free space, after Microsoft Windows has been 
installed and a permanent swap has been created. The system 
900 also includes either a 3-1/2 inch or 5-1/4 inch floppy 

25 disk drive (not shown) connected to the computer 902. Image 

files require, on the average, about one megabyte of storage 
space each. 

The system includes either a non-interlaced video monitor 
906 or an interlaced video monitor 908. If a user has a 
30 choice between a non-interlaced monitor 906 and the interlaced 

monitor 908 and both have equal resolution capability, the 
non-interlaced monitor 906 is preferred. A monitor designed 
to the VGA (video graphics array) display standard, or better, 
is required to use all the features of the imaging system 900. 



35 



The imaging system 900 includes, at a minimum, a 
Microsoft Windows-compatible, 256 -color Super VGA graphics 
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adapter or interface 910, having a memory (not shown) , 
connected to the computer 902 and to a monitor 906 or 908. 
The adapter 910 preferably has accelerated capabilities in 
Microsoft Windows for faster cind more efficient operation. 
5 For greater color capability, however, a Microsoft Windows- 

compatible True Color VGA adapter is preferred. 

An optional component of the imaging system 900 to 
capture video images for manipulation is a color video frame 
grabber interface or capture board 912 connected to the 

10 computer 902. Two such frame grabber interfaces are AITech 

and ComputerEyes RT. Typical video sources optionally 
connected to the frame grabber interface 912 include a video 
camera 914, a photo CD (Compact Disk) 916, a VCR (video 
cassette recorder) 918, a still video camera 920, a video 

15 camcorder (not shown) , or a laser disk player (not shown) . 

The preceding video sources are not intended to be the only 
ones that can be used, since other video sources that the user 
may have available could also be utilized. A procedure for 
capturing a video image using the frame grabber interface 912 

20 will be described hereinbelow. 

A keyboard 922 that is cotr^iatible with IBM AT computers 
is connected to the computer 902. A pointing device 924 
supported by Microsoft Windows such as any two or three button 
mouse also connects to the computer 902. Reference throughout 

25 the detailed description to use of the mouse is not meant to 

preclude use of another type of pointing device. 

Another optional way to capture a video image is by use 
of a scanner 926 connected to the computer 902. The imaging 
system 900 supports the following scanners: any Twain- 

3 0 compliant sccinner, such as the Logitech ScanMaui Color hand 

scanner; the Polaroid Digital Scanner SC-500; and the Toshiba 
SC-7500 scanner. 

The computer 902 connects to a printer 928 to provide a 
way to produce hard-copy output. The imaging system 900 

3 5 allows printing images to any graphics printer currently set 

up in Microsoft Windows with a Microsoft Windows driver, such 
as the Polaroid CI-700 printer. In addition, the imaging 
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system 900 provides internal printer drivers that allow 
printing from a plurality of color printer choices. 

The imaging system 900 requires Microsoft Windows version 
3.0 or higher and MS-DOS version 5 operating software. 
5 However, Microsoft Windows version 3.1 is preferred to obtain 

high quality, scalable fonts and the ability to rotate text 
objects smoothly. 

Several optional components that provide additional 
flexibility in use of the system 900 will now be described. 

10 To upload a video image or download an object or image file, 

a modem 930 interfaces between the computer 902 and a 
telephone line 932. By use of the modem 930, the user may 
have access to remote sources of video images. A video image 
can later be sent to a remote location via the modem 93 0 after 

15 the user has performed image manipulation. 

Another set of optional components connected to the 
computer 902 includes a server interface 934 which further 
connects to a server 93 6 and/or a print server 93 8. The print 
server 938 further connects to the printer 928. The server 

20 936 and print server 93 8 are typically part of a network such 

as a liAN (local area network) which is not shown. Use of a 
server 936 conveniently provides the user with greater storage 
capacity for raw aind final video images. The server 93 6 also 
provides a means of sharing objects and video images with 

25 other users of the server. The print server 938 provides a 

means to offload printing of video images from the computer 
902. The frame graibber 912 has a memory (not shown) in which 
an image may be manipulated at an industry standard video 
resolution (64 0 by 4 80 pixels) by the con^uter 902. Other 

30 video resolutions may be utilized, however. The frame grabber 

912 has a video port (not shown) that may be optionally 
connected to the printer 92B . 

The images utilized in the imaging system 90 0 have a 
background from which the user can select areas and objects to 

35 manipulate. Each of these image components will now be 

defined, along with manipulation of the components. 
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The background is the file that the user opens which 
serves as the backdrop of the image that will be manipulated. 
Areas of the background to manipulate are selected as 
discussed below. The user may also create objects from the 
5 background, as discussed below. 

An area is a portion of the background (or the entire 
background, if desired) that is selected for manipulation. 
The user can manipulate areas by using the options from Area 
or Brush Options, which will be described hereinbelow. For 

10 example, the user can paint on or adjust the color of an area. 

When painting or adjusting the color of an area, the size and 
shape of a paint brush is adjustable. The user can choose to 
change the foreground, middle ground, or background colors 
that are used for painting. Areas are stationary portions of 

15 the backgroiind and cannot be moved. 

Objects are the key to manipulating images with the 
imaging system 900 because of their versatility. Unlike 
areas, objects can be moved aroxind or over the backgroiind {or 
even into another file) , rotated, enlarged, reduced, layered, 

20 cloned, and even saved for future use. 

Once the user has selected an object, options from Object 
Options are used to manipulate the object. Text is also 
considered an object, so it can be manipulated in the same 
way. Manipulations to objects do not affect the background. 

25 If the user has selected more than one object, the 

imaging system 900 identifies the current object by 
surrounding it with a white box. Any object manipulations 
done are applied only to the current object. Other objects 
are still visible on the screen, but without the white border. 

30 The user can easily move from one object to another, making 

another object become the current object for manipulation. 

When the user is satisfied with how an object is 
manipulated, a copy of the object may be pasted to the 
background. At this point, the copy of the object becomes 

35 part of the background. It is no longer an object. The 

original object, however, remains on top of the newly pasted 
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copy. The user may continue to move and manipulate the 
original object or delete it. 

Any objects that are not pasted to the background will 
not be saved when the background file is saved unless 
specifically invoked by another file format. If the user 
wishes to save an object, it can be saved separately as an 
object. 
Glossary 

In view of the Windows environment that the system 900 
operates in, a glossary of Windows terminology is provided in 
Table 1. 

TABLE 1 

Choose To use a mouse or key combination to pick an item 
that begins an action. 



Click 

Dialog box 



To press and release a mouse button quickly. 

A rectangular box that either requests or 
provides information. Many dialog boxes 
present options to choose among before Windows 
can carry out a command. Some dialog boxes 
present warnings or explain why a command 
cannot be completed. 



Doub 1 e - c 1 i ck 



Drag 



Icon 



Menu 



Option 



To rapidly press and release a mouse button 
twice without moving the mouse. Double - 
clicking carries out an action, such as 
opening an icon. 

To move an item on the video screen by holding down 
a mouse button while moving the mouse. 



A graphical representation of an element 
software program or in Windows. 



Pixel 



A list of items, most of which are commands. Menu 
names appear in the menu bar near the top of the 
window. A command is used by selecting the menu 
name and then choosing the command. 

A choice in a dialog box. An option affects the 
way in which a command is caarried out. Dialog 
boxes have several kinds of options, including 
mutually exclusive option buttons and non-exclusive 
check boxes - 

The smallest graphical units on the video screen. 
Also known as picture elements (pels) . 
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Point To move the pointer on the video screen until it 

rests on the item desired to be selected or chosen. 

Pointer The arrow-shaped cursor on the screen that 
5 indicates the position of the mouse. 

Sub-menu A menu contained within another menu. 

Window A rectangular area on the video screen providing a 
10 view of an application or document - 

The menus, sub-menus, dialog boxes, icons and so forth in 
the Windows environment that are controlled by the keyboard 
922 (Figure 18) and pointing device 924 make up what is known 

15 as a user interface. By choosing a particular item in a 

dialog box, for example, the user controls a type, shape, or 
size of a manipulation tool that is visible on the video 
monitor 906/908. By another choice, the user can execute a 
particular command, operation or process. Thus, by using the 

20 Windows user interface to interact with the system 900, the 

user can manipulate or edit an image. 

A Clipboard is a temporary storage location that is 
available for transferring information. A Cut option and a 
Copy option are used to transfer information to the Clipboard, 

25 while a Paste option is used to transfer information from the 

Clipboard. The information is stored in the Clipboard until 
it is pasted or until another item (area or object) is cut or 
copied to the Clipboard. Only one area or object can be 
stored in the clipboard at a time. 

30 The Cut option has two variations: "Cut Area" places a 

selected area on the Clipboard without removing the selected 
area from an image, while "Cut Object" removes selected 
objects from the image and places the objects on the 
Clipboard. The Cut option requires at least one image to be 

35 open or active. Cutting to the Clipboard replaces the current 

contents of the Clipboard. This option is useful for moving 
areas or objects into another image. The Copy option is 
similar to the Cut option except that "Copy Object" does not 
remove the selected object from the image but instead makes a 

4 0 copy of the object. The Paste option also has two variations: 
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" Paste as New Image" inserts a copy of the Clipboard contents 
into a new image window, while "Paste as Object" inserts the 
contents of the Clipboard as an object into the active image. 
The Paste option is not valid if the Clipboard is empty. The 
5 user cam therefore use "Copy Area" to make a copy of a 

selected area and then use "Paste as Object" to paste the 
selected area as an object without having to first explicitly 
make the area an object. The original copied area remains an 
area, not an object. 

10 Referring generally to Figures 18, 19a and 19b, the top- 

level control flow 1002 performed by the system 900 will now 
be described. As previously mentioned, the flow 1002 is 
executed in the Windows environment on the computer 902. 

Beginning at a start step 1004, the computer 902 proceeds 

15 to a decision step 1006 to determine if an image is to be 

acquired by use of the scanner 926. If so, at step 1008, the 
image is scanned by the scanner 926 and then saved to a file 
with an associated filename provided by the user of the system 
900. The preferred location for saving the file is storage 

20 904, although the file may be saved elsewhere, e.g., server 

936, If the test at step 1006 proves false, the computer 902 
proceeds to a decision step 1010 to determine if an image is 
to be acquired by use of the frame grabber 912. If so, at 
step 1012, the video image is captured by the frame grabber 

25 912 and then saved to a file with an associated filename 

provided by the user of the system 900. Video images may be 
captured from a number of sources and later manipulated in the 
imaging system 900. For example, video images may be captured 
from the video camera 914, Photo CD 916, VCR tape 918, still 

3 0 video 920, or video camcorder (not shown) . The user of the 

system 900 may utilize none, some, or all of these video 
sources for capturing an image or sequence of images. 

The steps to capture a video image in the imaging system 
900 are as follows: 



35 



1. Click "File" on the imaging system 900 menu bar. 

2. Click "Acquire." 



wo 94/28490 PCTAJS94/05756 

-56- 

A sub menu appears on the video monitor screen. 

3 . Click "Video. . . " 

5 A dialog box appears on the video monitor screen. 

4. Click on the name of the video frame grabber 
interface installed in the computer and then click 
"OK. " 



10 



40 



A dialog box appears on the video monitor screen. 



5 . Turn on the video image source (VCR, Photo CD 
Player, Video Camera, Still Video, Laser Disk, 
15 Camcorder, or other analog video source) . 

The video image appears in the Preview window of 
the dialog box. 

20 6. Advance or reverse the video source until the frame 

desired for further editing or manipulation is 
reached. Then click on "Freeze." 

7 . If the frozen frame is the desired image for 
25 manipulation, click "OK." Otherwise, click on 

"Live" and repeat step 6. 

After clicking on "Freeze" and "OK," the dialog box 
closes, a new document window opens on the video 
30 monitor screen, and the video image selected 

appears in the new window. 

8. Save the new file as follows: 

3 5 Click "Save Image as..." on the File menu. A 

dialog box appears on the video monitor screen. 



Click on the drive aind directory (and sub 
directory, if applicable) desired to save to. 

Click inside the "File name" box in the dialog box. 



Type the name desired for this file. Use up to 
eight characters plus a period and a three letter 
45 extension that denotes a file format. Files saved 

by the system 900 are formatted in Targa True Color 
24 -bit {-TGA extension) format, e.g., DOGS.TGA, or 
another format as described hereinbelow. 

50 After saving the file at step 1012 or if the test at step 

1010 proves false, the computer 902 moves to a decision step 
1014 to determine if an upload operation is to be performed. 
If so, at step 1016, the computer 902 retrieves an image from 
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a remote location via the modem 930 and telephone line 932, 
and saves the image to a file as previously described. For 
example, a business may have several branch locations with 
computers having video images needed by the user of the system 
5 900. Step 1016 provides a quick way to retrieve these images. 

After saving the file at step 1016 or if the test at step 1014 
proves false, the computer 902 moves to a decision step 1018 
to determine if an image is to be imported from another 
program. These images may be formatted as Targa True Color 
10 24-bit (TGA) , BitMap (BMP) , Joint Photographic Experts Group 

(JPEG) , Tagged Image File Format (TIFF) , Microsoft Paintbrush 
(PCX), Photo Compact Disk (PCD), Graphics Interchange Format 
(GIF) '. Run Length Encoded (RLE) , Work in Progress (WIP) , or 
Compressed (CMP) files. If so, at step 1020, the user selects 
15 a TGA or other accepted format file to be imported into the 

system 900, If the decision step 1018 proves false, at step 
1022 the user selects an existing file to be opened by the 
computer 902. This file may be from a previous editing or 
manipulation session on the system 900, or as the result of 
20 step 1008, 1012, or 1016. Nevertheless, at the conclusion of 

step 1020 or step 1022, the computer opens an image file and 
proceeds to the off -page connector D 1024 to resume the flow 
on Figure 19b at a decision step 1026. 

At step 1026, a determination is made whether the user 
25 desires to work with an area of the image. If so, the 

computer 902 moves to a decision step 1028 to begin a series 
of area choices that are presented to the user. If the user 
does not desire to perform an area operation as determined by 
step 1026, the computer 902 moves to a decision step 1044 to 
30 determine if the user desires to work with an object. 

Returning attention to decision step 1028, the computer 
902 determines if the user desires to perform a mask 
procedure. If so, the computer 902 proceeds to a mask 
function 1030. A mask is an advanced feature of the imaging 
35 system 900 offering the user great flexibility in manipulating 

color in images by choosing which existing colors in the image 
will or will not be affected by further color manipulations. 
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The mask function 1030 will be further described in 
conjunction with Figure 20. 

Upon return from the mask function 1030, or if the 
decision step 1028 is false, the computer moves to a decision 
5 step 1032 to deteinnine if the user desires to perform a top- 

level gamma procedure- If so, the computer 902 proceeds to 
the top-level gamma function 262. The function 262 was 
previously described in conjunction with Figure 12. Upon 
return from the top-level gamma function 262, or if the 
10 decision step 1032 is false, the computer moves to a decision 

step 1034 to deteirmine if the user desires to perform one of 
the area options. If so, the computer 902 proceeds to step 
1036 wherein one or more area options are performed. The area 
options will be described hereinbelow. 
15 A description of how an area is selected follows. The 

presently preferred process of selecting an area of the 
background image to be maniptilated is defined has eight 
options: Drag Box, Drag Free, Area Addition, Color Select^ 
Brush Select, Area Subtraction, None, and Inverse. Of course, 

2 0 other embodiments of the invention may include other area 

selection methods. Drag box, drag free, and other pointing 
device selection methods are commonly called "mouse events", 
as the mouse is a very popular example of the pointing device 
924 . A selected area is indicated on the video monitor 
25 906/908 by an animated black and white marquee that borders 

the selected area. Descriptions of these options follow: 

Drag Box: Allows the user to create a box boundary around 
part of an image for manipulation only within that selected 

3 0 area , 

Freehand Drag: Allows the user to select an area of the image 
with a freehand outline or a series of connected lines . To 
create a freehand outline or boundary, drag the mouse and 
35 double click when finished. To create a connected- line 

outline, click once, position the mouse pointer at the next 
. point, click again, and so on, until all lines have been 
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drawn; then double click to complete the selection. The 
bovmdary created by the connected lines defines a polygon. 

Area Addition: Allows the user to add additional selections 
5 to the current selection. Once the user clicks the area 

addition tool, "add" mode is retained until it is clicked 
again or switched to a different selection tool (drag box, 
drag free, color select, or brush select) • All areas selected 
are outlined with the animated black and white marquee. 



10 



15 



20 



30 



Color Select: Allows the user to select all contiguous pixels 
of the color that is clicked on. For example, having a red 
shape surrounded by another color, if the user clicks on the 
red shape, all contiguous red pixels are selected. 

Brush Select: Allows the user to select an area using the 
brush. The pixels that are electronically "touched" by the 
brush that is controlled by the user using the pointing device 
924 are selected. 



~ Area Subtraction: Allows the user to deselect certain regions 
of the currently selected area. Click on the Area Subtraction 
tool and select areas to be subtracted. Again, "subtract" 
mode is retained until it is clicked again or switched to a 
25 different selection tool (drag box, drag free, color select, 

or brush select) . 

None: Clears all current area selections and selects the 
entire image for manipulation. 



Inverse: Reverses the current selection. All currently 
selected areas become unselected and all previously deselected 
areas become selected. 



35 



Upon return from the area options step 1036, or if the 
decision step 1034 is false, the computer 902 moves to a 
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decision step 1038 to determine if any alpha bits or flags are 
set in the alpha channel. If so, the computer advances to 
step 1039 and adjusts the available option set of subsequent 
operations. Certain options of an option family, e.g, brush 
5 options, are available or performed only if a particular alpha 

bit that corresponds to that option is set. Alpha channel 
bits were previously described in conjunction with layering 
(Figure 11) and transparency/opacity (Figure 16) . 

Upon return from the alpha channel step 103 9, or if the 

10 decision step 1038 is false, the computer 902 moves to a 

decision step 1040 to determine if the user desires to perfoinn 
one of the brush options. If so, the computer 902 proceeds to 
step 1042 wherein one or more brush options are performed. 
The brush options will be described hereinbelow. A link (not 

15 shown) exists between the brush options 1042 and the area 

options 1036 to allow the option groups to work together, such 
as the brush shape being a small area of itself, A similar 
link (not shown) exists between the brush options 1042 and 
object options 1052 to allow those option groups to work 

20 together, for such cases where a brush option is only effected 

on an object. 

Upon return from the brush options step 1042, if the 
decision step 1040 is false, or if the decision step 1026 is 
false, the conputer moves to decision step 1044 to determine 

25 if the user desires to work with an object. If so, the 

computer 902 moves to a decision step 1046 to begin a series 
of object choices that are presented to the user. If the user 
does not desire to perform an object operation as determined 
by step 1044, the computer 902 moves to a decision step 1066 

3 0 to determine if the user desires to make any additional 

selections. 

Returning attention to decision step 1046, the computer 
902 determines if any alpha bits or flags are set in the alpha 
channel. If so, the computer advances to step 1047 and 
35 adjusts the available option set of subsequent operations. 

Moving to a decision step 1048, the computer 902 determines if 
the user desires to perform the mask procedure. If so, the 
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computer 902 proceeds to the mask function 1030, The mask 
function 1030 will be further described in conjunction with 
Figure 20. 

Upon return from the mask function 1030, or if the 
5 decision step 1048 is false, the computer moves to a decision 

step 1050 to determine if the user desires to perform one of 
the object options. If so, the computer 902 proceeds to step 
1052 wherein one or more object options are performed. The 
object options will be described hereinbelow. A link (not 

10 shown) exists between the object options 1052 and the area 

options 1036 to allow the option groups to work together, such 
as with the option "Object Re-color" where an object is used 
to affect an area. Upon return from the object options step 
1052, or if the decision step 1050 is false, the computer 

15 moves to a decision step 1054 to determine if the user desires 

to perform a top-level gamma procedure. If so, the computer 
902 proceeds to the top-level gamma function 262. The 
fxinction 262 was previously described in conjunction with 
Figure 12. Upon return from the top-level gamma function 

20 262, or if the decision step 1054 is false, the computer moves 

to a decision step 1058 to determine if the user desires to 
perform layering. If so, the computer 902 proceeds to the 
layering function 274. The layering function 274 was 
previously described in conjunction with Figure 11. 

25 Upon return from the layering function 274 or if the 

decision step 1058 is false, the computer moves to decision 
step 1062 to determine if the user desires to work with text. 
If so, the computer advances to a step 1064 wherein text is 
added, deleted, or otherwise manipulated. Options available 

3 0 include changing font, font size, font style; regular type, 

bold type, shadow, keyline, 3-dimension; centering 
horizontally, centering vertically ; left justification, center 
justification; restore; transparent text background; moving 
text; sizing text; and rotating text. Keyline formats the 

35 text in a typeface that incorporates into each letter the 

foreground, middle ground, and background colors selected by 
the user. Since text is considered an object by the system 
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900/ the move, size, and rotate options basically are the same 
as those in the object options step 1052. Text objects 
default to black with a transparent background. Note that 
text rotation requires, at a minimum, Microsoft Windows 
5 revision 3.1 and True Type fonts. 

Upon completion of the text step 1064, if decision step 
1062 is false, or if the decision step 1044 is false, the 
computer 902 moves to a decision step 1066 to determine if the 
user desires to perform additional area or object 

10 manipulations. If so, the computer 902 loops back to step 1026 

to repeat the steps 1026 through 1066. If no additional 
manipulations are desired, as determined by step 1066, the 
computer 902 advances to a decision step 1068 (via the off- 
page connector E 1067 to Figure 19c) to determine if the user 

15 desires to print the current image. If so, at step 1070, the 

computer 902 sends the current image to be printed at the 
printer 928. This image is a digital image from the computer 
902 or the storage 904 at a user-selectable resolution. In 
another embodiment, the cotrputer 902 sends the current image 

20 through the server interface 934 to the print server 938 for 

subsequent printing at the printer 928. In yet another 
embodiment that is not shown on Figure 18, the frame grabber 
912 can send a video image, at video resolution, via the video 
out port to the printer 928. 

25 Upon completion of the printing step 1070 or if decision 

step 1068 is false, the computer 902 moves to a decision step 
1072 to determine if the user desires to perform a download 
operation. If so, at step 1074, the computer 902 sends the 
current image to a user- determined destination via the modem 

30 930 or to the server 936. The download operation can be done 

only if the modem 930 or server 936 are connected to the 
system 900. 

Upon completion of the download step 1074 or if decision 
step 1072 is false, the computer 902 moves to a decision step 
35 1076 to determine if the user desires to save the current 

image in a file. If so, at step 1078, the user chooses among 
several options. If the user created the current image, e.g. 
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by drawing, the "Save Image As ..." selection prompts the user 
for a path and filename in which to save the file, and the 
number of colors with which to perform the save. After the 
user enters the requested information, the computer 902 
5 performs the save. To save changes to a file that has 

previously been saved, the user selects "Save Image". To save 
changes to a file while keeping a copy of the original 
version, the user selects "Save Image As ..." and provides a 
filename that is different than the original version. Only 

10 the image background is saved. Any objects not affixed to the 

background are not saved. To save an image and its objects 
without affixing the objects to the background, the user 
selects "Save Work in Progress" and provides a path and 
filename. The .WIP extension is used with saving all work in 

15 progress. To save a current object that has been created, the 

user selects "Save Object..." and provides a path and 
filename. Object images are saved with a file extension of 
.IMI, while pictvire or background images are saved with a file 
format (e.g., TGA) as described in conjunction with step 1018. 

20 Three formats in addition to those previously described. 

Encapsulated PostScript (EPS) , Windows Metafile (WMF) , and 
Imagewear (PAM) , are only available for saving (write only) . 

Upon completion of the save step 1078, or if decision 
step 1076 is false, the computer 902 moves to step 1080 to 

25 close the current file, which removes it from the video 

monitor screen. The top-level control flow 1002 ends at step 
1082. 

Referring to Figure 20, the color mask function 1030 
defined in Figure 19b will now be described. The masking 

30 feature of the imaging system 900 provides the flexibility in 

manipulating color in images by choosing which existing colors 
in an image will or will not be affected by further color 
manipulations. The mask allows changes to be made based on 
color instead of area. When a mask is in use, manipulations 

35 affect only those colors defined in the mask, unless the 

Invert option is selected. Using the Invert option, all 
colors of the image except those defined to be in the mask 



wo 94/28490 PCTAOS94/05756 

-64- 

will be affected by further manipulation. Only one mask is in 
use at a time. A mask applies only to the active image 
window. 

Beginning at a start step 1102, the computer 902 proceeds 
5 to determine if the user desires to create a mask at a 

decision step 1104. If so, the computer 902 moves to a 
decision step 1108 and determines if the user desires to begin 
with an empty mask, i.e., no colors are in the mask. Each 
non-object image has an associated mask that originally is 

10 available when the image is created, and the mask is in the 

empty condition. If the user does not desire to begin with an 
empty mask, the computer moves to step 1109 wherein all the 
colors of the image are captured on a pixel by pixel basis by 
the coir^uter 902 and placed in the current mask. Color 

15 capture is described at function 1110 (Figure 21) below. 

However, if the user desires to begin with an empty mask as 
determined by step 1108, the computer 902 proceeds to a mask 
color structure function 1110 wherein the user selects the 
colors to be affected or iinaffected by further color 

20 manipulations- The user perfoirms this selection by any of the 

methods previously described for selecting an area, e.g.. Drag 
Box. A color value entry is then made in the mask color 
structure by the computer 902 for each color selected. 
Function 1110 is further described in conjunction with Figxire 

25 21. Moving to a decision step 1114, a determination is made 

whether the user desires to select any additional mask 
options- If so, the computer 902 moves to a decision step 
1132. However, if no additional options are desired by the 
user as determined in step 1114, the computer 902 moves to 

30 step 1116 wherein the user selects a "Turn Mask On" option . 

The word "Mask" appears on the lower portion of the video 
monitor 906/908 to indicate that the mask is in effect or 
activated. The computer 902 returns at step 1118 to the top- 
level control flow 1002 (Figure 19) . 

35 The user is now ready to use the mask. The user selects 

an area, using any of the area selection methods previously 
mentioned, or selects an object for color manipulation. The 
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area can be the entire image or a portion of the entire image. 
Object selection will be described in conjunction with the 
description of the object options hereinbelow. When the mask 
is activated, any changes made using the following options • 
5 affects only those colors identified in the mask: Brush 

options; Area and Object options of Fill, Tint, Posterize, 
Gray-scale, Sharpen, Blur, and Adjust Color; and Area option 
Stamp into Selection, These Brush, Area, and Object options 
are described hereinbelow. Colors of the image that are not 

10 included in the mask are not changed. The image manipulation 

performed by these options on the pixels that correspond to 
the colors identified in the mask is done automatically for 
all the colors of the mask, i.e., the user does not repeat the 
option individually for each color in the mask. Once the user 

15 selects the colors desired for the mask and initiates one of 

the above-listed image manipulation options, the computer 902 
processes all the colors of the mask without further user 
intervention . 

The system 900 provides several commands for the user to 
20 obtain mask information while creating the mask. These are 

described as follows: 

Change to White, Colors in Mask: All the colors selected 
for the mask appear in white on the image. The 
user thus sees how much of the image is impacted by 
25 the mask colors. 

Change to White, Colors Not in Mask: All the colors 
not selected for the mask appear in white on the 

image, or in other words, the image is displayed with 

3 0 only the colors selected for the mask and all other 

colors are shown in white. The user thus sees how 
much of the image is not affected by the mask 
colors . 

35 Normal Mask View: This command cancels either of the 

Change to White commands above , The image is shown 
with normal colors, i.e., the white covering is 
removed . 

40 View Outline of Mask Colors: An outline of alternating 

yellow and black dashes is displayed around all 
instances of the colors selected for the mask 
within the image. 
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Remove Outline of Mask Colors: This command takes away 
the mask color outline of the View Outline command. 

Preview Mask: A View Mask box is displayed on the video 
5 monitor 906/908 so that the user can quickly and 

easily see, before actually using the mask, if all 
the colors desired for the mask have been selected. 
Thus, the user can check the colors selected so far 
for the mask, and determine, for example, if too 

20 many colors were inadvertently selected. The left 

side of the box displays a preview of the entire 
image. A selection box below the preview image 
allows the user two choices of the colors to view. 
When "In Mask" is selected, only the colors 

15 selected for the color mask are displayed in color. 

All other colors of the image are displayed in 
white. When "Not in Mask" is selected, only the 
colors not in the mask are displayed. Colors in 
the mask are displayed in white. The right side of 

20 the View Mask box displays a palette or spectrum of 

colors that are in , or not in , the color mask , 
depending on whether "In Mask" or "Not in Mask" is 
selected. 

25 Referring again to Figure 20, if the decision step 1104 

proves false, i.e., the user does not desire to create a mask, 
or if the user desires an additional mask option, as 
determined by step 1114, the computer 902 proceeds to 
determine if the user desires to add additional colors to the 

30 mask at the decision step 1132. If so, the computer moves to 

the mask color structure function 1110 wherein the user 
selects the colors to be added to the mask for further color 
manipulations. Function 1110 is further described in 
conjunction with Figure 21. Moving to a decision step 1140, 

35 a determination is made whether the user desires to select any 

additional colors to be added. If so, the computer 902 loops 
back to fixnction 1110 to repeat the function for the 
addi t ional select ion . 

However, if the decision step 1140 step proves false, 

40 i.e., when no further colors are to be selected for the mask, 

the computer 902 moves to a decision step 1144 wherein a 
determination is made whether the user desires to select any 
additional mask options. If so, the computer 902 moves to 
decision step 1132, However, if no additional options are 
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desired by the user as determined in step 1144, the computer 
902 moves to step 1116 wherein the user sets the "Turn Mask 
On" option. The word "Mask" appears on the lower portion of 
the video monitor 906/908 to indicate that the mask is in 
5 effect. The computer 902 returns at step 1118 to the top- 

level control flow 1002 (Figure 19) . As previously described, 
the user is now ready to use the mask. 

However, if the decision step 1132 proves false, i.e., 
the user did not desire to add colors to a previously defined 

10 mask, the computer 902 proceeds to determine if the user 

desires to subtract or delete colors from the mask at a 
decision step 1146. If so, the computer executes function 
1110 and a set of steps 1154, 1158 that are the same as 
function 1110 and steps 1140, 1144 except that function 1110 

15 and step 1154 subtract or delete colors from the mask rather 

than adding colors. Also, at function 1110, a color value is 
deleted from the color structure by the computer 902 for each 
color selected . 

However, if the decision step 1146 proves false, i.e., 

20 the user did not desire to subtract colors from a previously 

defined mask, the computer 902 proceeds to determine if the 
user desires to load a previously defined mask at a decision 
step 1160. If the mask for the current image is not empty, 
i.e., has colors in it, at the time of the load, the colors in 

25 the mask are replaced by those in the mask that is loaded. If 

decision step 1160 is true, the computer 902 moves to step 
1162 wherein the user mouse clicks on a "Load Mask. . option. 
The computer 902 then displays a dialog box on the video 
monitor 906/908 showing a list of previously created mask 

3 0 files from which to choose. The user can change the drive 

and/or directory to see a list of previously created mask 
files on different drives and/or directories. At step 1164, 
the user selects a mask file to be loaded and mouse clicks on 
the file name. Moving to step 1166, the user sets the "Turn 

3 5 Mask On" option. The word "Mask" appears on the lower portion 

of the video monitor 906/908 to indicate that the mask is in 
effect. The computer 902 returns at step 1118 to the top- 
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level control flow 1002 (Figure 19) . As previously described, 
the user is now ready to use the mask. 

However, if the decision step 1160 proves false, i.e., 
the user did not desire to load a previously defined mask, the 
5 computer 902 proceeds to determine if the user desires to 

disable the most recently defined or current mask at a 
decision step 1170. If so, the computer 902 moves to step 
1172 wherein the user sets a "Turn Mask Off" option. This 
capability is for when the user does not wish to use the most 

10 recently defined mask, but may use the mask later, or if the 

user is finished using the mask. The "Mask" indicator 
disappears from the bottom of the video monitor screen to 
indicate that no mask is in use. The computer 902 maintains 
the selected colors in the mask color structure until the user 

15 terminates the image editing program, or the user selects the 

create mask option at steps 1104 to 1114 (which begins the 
color selection process anew) . In either of the two cases of 
the last sentence and if the user has not saved the mask to a 
file, the colors selected for the mask are lost. The computer 

20 902 returns at step 1118 to the top-level control flow 1002 

(Figure 19) - 

If the decision step 1170 proves false, i.e., the user 
did not desire to disable the current mask, the computer 902 
proceeds to determine if the user desires to enable the most 

25 recently defined mask at a decision step 1174. If so, the 

computer 902 moves to step 1176 wherein the user sets the 
"Turn Mask On" option. Then, "Mask" appears on the lower 
portion of the video monitor screen to indicate that the mask 
is in effect. The computer 902 returns at step 1118 to the 

30 top-level control flow 1002 (Figure 19) . As previously 

described, the user is now ready to use the mask. 

If the decision step 1174 proves false, i.e., the user 
did not desire to enable the current mask, the computer 902 
proceeds to determine if the user desires to save the most 

35 recently defined mask at a decision step 1178. The user may 

want to save the mask, and thereby save the colors in the 
mask, if there is a chance of reusing the mask at a later 
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time- If so, the computer 902 moves to step 1180 wherein the 
user mouse clicks on a "Save Mask, . selection. The computer 
902 then displays a dialog box on the video monitor 906/908. 
The user can change the drive and/or directory to which the 
5 created mask will be saved and then type the desired name to 

correspond with the file using up to eight characters plus an 
extension of ".MSK" to identify this file as a mask file. The 
computer 902 returns at step 1118 to the top-level control 
flow 1002 (Figure 19) . The save capstbility allows the user to 

10 keep a current copy of a recently created mask, which may 

often be reused in the future, as a file. In this way, the 
user can then load the mask with the same or any other image 
file desired. The user can save as many masks as desired. 

If the decision step 1178 proves false^ i.e., the user 

15 did not request a save of the current mask, the computer 902 

proceeds to determine if the user desires to invert the 
current mask at a decision step 1182, If so, the computer 902 
moves to step 1184 wherein the computer performs an invert of 
the mask. The invert operation results in the mask having all 

20 the colors of the image except those that were defined to be 

in the mask before step 1184. At the completion of step 1184 
or if decision step 1182 proves false, the computer 902 
returns at step 1118 to the top-level control flow 1002 
(Figure 19) . 

25 Referring to Figure 21, the mask color structure defined 

in block 1110 of Figure 20 will now be described. Beginning 
at a start step 1202, the computer 902 moves to step 1204 
wherein the user selects an area of the current image to 
identify a color or colors of interest. This selection is 

30 accomplished by any of the area selection methods previously 

described. The selected area is surrounded by an outline of 
alternating dashes. By use of a zoom option, the area 
selected may be as small as a single pixel . 

Moving to step 1206, the computer begins at a preselected 

35 pixel location of the selected area. Then, at step 1208, the 

computer 902 determines the color and value of the current 
pixel. The value, for example, may include hue and saturation 
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attributes or components, and will be described hereinbelow. 
Moving to a decision step 1210, the computer 902 tests whether 
the color value for the color of the current pixel as 
determined at step 1208 is already linked in a linked list 
5 structure of color values. If not, the computer 902 adds the 

color value for the color of the current pixel to the mask 
color structure at step 1212. The mask color structure is 
further described below. At the completion of step 1212, the 
computer 902 moves to step 1214 wherein the color value for 
10 the color of the current pixel is linked to a previous color 

value for another color (if one exists) of the current mask by 
use of the linked list structure. There is a linked list 
structure for each mask (current or saved as a file) . Linked 
list structure, along with linking and unlinking procedures, 
15 is well known in the software art. 

However, if the decision step 1210 proves true, i.e., the 
color value of the current pixel is already in the linked 
list, the computer 902 proceeds to a decision step 1216. At 
step 1216, the computer 902 determines if the subtract option 
20 was selected in the calling color mask function 103 0 (Figrure 

20) . If so, at step 1218, the computer 902 deletes the color 
value from the color stiructure and also removes the color 
value from the linked list. 

Upon completion of either step 1218 or step 1214, or if 
25 the decision step 1216 proves false, the computer 902 moves to 

a decision step 1220 to determine whether there are additional 
pixels in the selected area to be tested. If so, at step 
1222, the computer 902 advances to the next pixel in the 
selected area and loops back to step 1208 to determine the 
3 0 color and value of the new current pixel . The loop of steps 

1208 through 1222 continues until the decision step 1220 
determines that all pixels in the selected area have been 
tested, at which time the computer returns at step 1224 to the 
color mask function 1030 (Figure 20) . 
35 Figures 22a and 22b illustrate an exemplary mask color 

structure 1302. The mask color structure provides a way of 
visualizing the colors and their attributes. As seen in 



wo 94/28490 PCT/US94/a5756 

-71- 

Figure 22a, the vertical axis 13 04 represents the contrast 
attribute in a range from white at the top to black at the 
bottom. On the axis 13 04 are three disks representing a red 
plane 1306, a green plane 1308 and a blue plane 1310. A 
5 single plane, for example, the green plane 1308, is shown in 

Figure 22b. The disk of the green plane represents hue 
attributes and saturation attributes for this plane. The hue 
attribute value has a numerical range of zero to 255 starting 
at an origin 1320 and proceeding counter-clockwise around the 
10 disk. 

For the color under consideration, green, for example, 
values near the zero end of the scale denote less green, while 
values near the 255 end of the scale denote more green. The 
saturation attribute value also has a numerical range of zero 

15 to 255 starting at an origin 1322 at the center of the disk 

and proceeding radially outward to the edge of the disk. An 
example of a pixel having a hue attribute value of 64 and a 
saturation attribute value of 170 on the green plane 1308 is 
identified by a small box 1324 on both Figures 22a and 22b. 

20 A second small box 1326 represents a slightly different but - 

unique color than that of box 1324. Using this model, over 16 
million unique colors can be represented. Of course, other 
color models can be used in representing a color structure. 

Each color that is defined to be part of a mask is 

25 identified on the color structure and is also linked with a 

previous color (if one exists) in the linked list. When a 
mask is utilized for pixel manipulation, mask processing of 
the colors is achieved by use of the linked list of the 
defined colors. When the user has finished selecting the 

30 colors desired for the mask and initiated an operation using 

the mask, the computer 902 sequentially processes each color 
in the linked list, without further user intervention. 
Application of Masks 

As an example, consider an image acquired by use of the 

35 frame grabber interface 912. This image has a background and 

a person in the foreground. When this image was digitized 
prior to being stored by the computer 902, video noise was 
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also digitized and stored. Video noise includes extraneous 
signals or unwanted disturbances that appear on a video image 
due to the recording (loss of generation signal) and 
digitizing (analog degradation) processes. To remove the 
5 pixels of video noise from the image, the user defines a mask 

to include the colors of the video noise. The user then 
selects a Blur option using the mask to target the noise 
pixels to be blended into the desired surroiinding background 
image. The blending of the noise pixels effectively removes 

10 the video noise. 

Use of the mask provides great capability in editing 
sequences of frames. These sequences can be, for example, 
from a roll of film used in a still camera and transferred to 
a Photo CD, or from a camcorder video. Each image frame is 

15 processed by the steps of Figure 19 in a sequential manner and 

* then a loop (not shown) is done to access the next frame and 
repeat the process until the sequence of frames is completed. 

As an example, a video sequence of frames acquired from 

20 a camcorder has images of a scenic background and a person. 

A mask is defined to include the background. The user can 
choose to entirely replace the background or have a special 
effect added by use of the mask. The person is not affected 
by the changes. This procedure allows the user to change or 

25 modify the backgroTind of the camcorder video sequence without 

affecting the image of the person. The preceding example can 
be reversed such that the background is retained and the mask 
is defined for the colors of the person. In this case, the 
user desires to keep the background as is, but desires to 

3 0 modify or replace the person with another person or object. 

As another example, consider a photographer who used a 
roll of daylight type film exposed under fluorescent lighting 
conditions. This roll of film is transferred to a Photo CD 
and then acquired by the computer 902. The user of the system- 

35 900 can remedy the lighting problem by defining a mask for the 

color of the fluorescent lighting and then removing the color 
from each frame of the film. The resultant frames are free of 
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the negative effects of the fluorescent lighting. Object 
Re Color 

One of the options available at the brush options step 
1042 (Figure 19b) is Object Re-Color, Referring to Figure 23, 
5 a description of the Object Re-Color option 1400 follows. 

Beginning at a start step 1402, the computer 902 advances 
to a decision step 14 04 to determine whether the user desires 
to use the "Grab Color" method. If so, the computer moves to 
step 1406 wherein the user selects either a single pixel or an 
10 area on the image. Then, at step 1408, the computer grabs the 

color to be used in subsequent processing from the selection 
of step 1406. If the computer determines, at a decision step 
1410, that only one pixel was selected at step 1406, the color 
is defined (by step 1408) , and the computer moves to step 
15 1422. 

However, if more than one pixel was selected at step 
1406, the computer 902 moves to step 1412 and begins, at a 
predetermined pixel location of the selected area, a process 
to determine an average hue (steps 1414-1420). At step 1414,- 

20 the computer 902 determines the hue of the current pixel. 

Moving to a decision step 1416, the computer determines if the 
last pixel of the selected area has been processed. If not, 
the computer 902 moves to step 1418 and advances to the next 
pixel in the selected area. The computer 902 then moves back 

25 to step 1414 to repeat the hue determination process for the 

next pixel. If the last pixel has been processed, as 
determined by step 1416, the coir^uter 902 moves to step 1420 
and calculates the average hue of the pixels in the selected 
area . 

30 At the completion of step 1420, or if only one pixel was 

selected, the hue of the single pixel or the average hue has 
been determined. At step 1422, the user selects a brush size 
and shape via the Windows user interface, along with an object 
that is to be changed by option 1400. At step 1424, the hue 

35 of the desired object is changed to the above -determined hue, 

or in other words, the object is re -colored. The luminance 
and contrast of the object are not affected, such that the 
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monochrome content remains the same. For example, the user 
could change the hue of a three dimensional box from yellow to 
red. Re -Color acconplishes this hue change without affecting 
the shadows, texture, attributes, and artifacts of the box. 
5 A texture operation is optionally performed by the computer 

902 at step 1448, which is described hereinbelow. The 
computer 902 returns at step 1426 to the brush options step 
1042 (Figure 19b) . 

However, if the user does not desire to use the "Grab 

10 Color" method, as determined at decision step 1404, the 

computer moves to step 143 0 wherein the user selects an area 
of the image to be used as an object. This selection is done 
by any of the area selection methods previously described. At 
step 1432, the user selects a "Grab Object" option so that the 

15 computer 902 grabs the selected area as the desired object. 

Then, at step 1434, the computer 902 begins a process at a 
predetermined pixel location of the object to determine an 
average hue. At step 1436, the computer 902 determines the 
hue of the current pixel. Moving to a decision step 1438, the 

20 computer determines if the last pixel of the object has been 

processed- If not, the computer 902 moves to step 144 0 and 
advances to the next pixel in the selected area. The computer 
902 then moves back to step 1436 to repeat the hue 
determination process for the next pixel. If the last pixel 

25 has been processed, as determined by step 1438, the computer 

902 moves to step 1442 and calculates the average hue of the 
pixels in the object. 

At the completion of step 1442, the average hue of the 
object has been determined. At step 1444, the user may either 

30 select another object or create a new object. Then, at step 

1446, the computer 902 changes the hue, or re -colors the 
object selected at step 1444 to the average hue, determined at 
step 1442, of the previously selected object. For example, if 
the background image is a blue sky with a cloud, the user 

35 could grab the cloud as an object. The system 900 derives an 

average hue of the cloud object. This average hue can then be 
used to create additional clouds of a matching hue. At step 
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144 8, an optional Object Texture Recolor operation is 
performed that textures an area using the color of the object 
identified at step 1432. Finally, the computer 902 returns at 
step 1426 to the brush options step 1042 (Figure 19b) . 
3 Additional options that are implemented in the system 90 0 

include the following. The options are grouped into the Area 
Options (step 1036 of Figure 19b) , Brush Options (step 1042) , 
and Object Options (step 1052) - 



10 Area Options 

The area options step 103 6 of Figxire 19b includes a 
plurality of options to manipulate areas of the background. 
Some of the options that the user may wish to perform on an 
image using a mask are in the area options grouping. The area 

15 options are summarized as follows: 

Sharpen: Increases the contrast between the adjacent 
colors in the selected area to more sharply define the area. 
A sub-menu is displayed on the video monitor 906/908 which 
20 allows the user to choose the level of sharpening from among 

low, medium, or high. 

Blur: Blends the edges between one color and another 

to create a soft effect- A sub-menu is displayed which allows 
25 the user to choose the level of blurring from among low, 

. medium, or high. 

Fill: Fills the selected area with color. A dialog 

box is displayed which allows a choice of whether to fill the 
30 area evenly or on a gradient. 

Tint: Applies a shade of the foreground color to the 

selected area that replaces the hue and saturation of the area 
with that of the tint applied, while maintaining the lightness 
35 of the area. 
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Posterize; Reduces the number of colors in the 

selected area. A sub-menu is displayed which allows the user 
to choose how many colors are to be used in the area. 

5 Gray-Scale : Transforms the selected area to gray- 

scale by averaging the reds, greens, and blues in the selected 
area . 

Adjust Color: Adjusts the existing color in the 
10 selected area. A dialog box is displayed which gives the user 

a choice of whether to adjust the color evenly or on a 
gradient. Then, a choice of whether to adjust the values of 
RGB (red, green, blue) , HSL (hue, saturation, lightness) , or 
HSC (hue, saturation, contrast) is given- Fiarther choices are 
15 provided based on the responses to the above determinations. 

In other embodiments, color models such as CMYK (cyan, 
magenta, yellow, black) and YUV (chroma, contrast, intensity) 
may be utilized. 

20 Flip Horizontal: Turns a selected area 180 degrees 

horizontally to create a mirror image of the original. If no 
area is selected, the entire image is flipped. 

Flip Vertical: Turns a selected area 180 degrees 
25 vertically such that the bottom of the picture becomes the 

top. If no area is selected, the entire image is flipped. 

Stamp into Selection: Stamps or clones copies of the 
current object into a selected area. The user must create a 
3 0 new object or load an existing object before using this 

option. The object is stamped until the selected area is 
filled. If no area selection is made, copies of the object 
are stamped into the entire image. 

The Stamp into Selection option can be used to tile a 
35 border around an image. First the user loads or creates an 

object. Then, by using an area selection option, e.g., area 
• box, the user draws a box in the center of the image at a 
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distance of a desired border width from each edge of the 
image. Next, the user chooses the invert option to select the 
area of the image outside the area box that will contain the 
border. Finally, the user chooses the Stamp into Selection 
5 option to clone the object in a tile pattern (the entire 

object is repeated such that the objects are adjacent) in the 
border area. Thus, the original image remains in the center 
area, and a border having a repeating pattern of the current 
object is created around the center image. 



Brush Options 

The brush options step 1042 of Figure 19b includes a 
plurality of options to manipulate areas of the background. 
15 However, some options use objects during the execution. Some 

of the options that the user may wish to perform on an image 
using a mask are in the brush options grouping. The brush 
options are summarized as follows: 

20 Sharpen: Increases the contrast between adjacent colors 

to more sharply define an area as the user holds down the left 
mouse button and drags the mouse . 

Blur: Blends adjacent colors to soften an area as the 

25 user holds down the left mouse button and drags the mouse • 



"Painting and Drawing" Group of the Br ush Options 

The following brush options from the brush options step 
104 2 are used for drawing or to paint over an area. These 

3 0 options affect background and selected areas only, not 

objects. A paintbrush that is controlled by the user using 
the pointing device 924 (Figure 18) is adjustable in size and 
shape from among many size selections, including eight 
circular sizes, six square sizes, two diagonal lines, four 

35 lengths of horizontal lines, and four lengths of vertical 

lines. These selections are made using the Windows user 
interface. The user may also choose particular colors by use 
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of the user interface . The foreground, middle ground, or 
background color that is used for painting may be chosen by 
the user from a palette of colors. The user then may choose 
between the foreground, middle ground, or background color by 
5 use of the pointing device 924. For example, using a three 

button mouse, the left mouse button applies the foreground 
color, the middle button applies the middle ground color, and 
the right mouse button applies the background color. 

10 Freehand Draw: Allows the user to draw freehand design 

as the mouse button is held down and the mouse is dragged. 

Line Draw: Allows the user to draw a perfectly 

straight line by clicking a mouse button at the beginning of 
15 the line and again at the end of the line. 

Connected Line Draw: Allows the user to draw a 

series of connected lines. This option is similar to "Line 
Draw" except that the user clicks with a different mouse 
20 button when finished. 

Frame: Allows the user to draw a rectangular frame on 
the image. To draw a frame in the shape of a true square, the 
user holds the CTRL key down while drawing. 

25 

Filled Frame: Allows the user to draw a rectangular 
frame filled with solid color. To draw a filled frame in the 
shape of a true square, the user holds the CTRL key down while 
drawing . 

30 

Air Brush: Allows the user to draw with a spray 

paint effect . 



35 



Round Cornered Frame; Allows the user to draw a frame 

with rounded comers. To draw a true square, the user holds 
the CTRL key down while drawing. 



4 
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Filled Round Cornered Frame : Allows the user to draw a 
frame with rounded corners filled with solid color. To draw 
a true square, the user holds the CTRL key down while drawing. 

5 Flood Fill: Replaces all adjacent areas of the exact 

color the user clicks on in the image with the foreground, 
middle ground, or background color, depending on whether the 
user clicks the left, middle, or right mouse button, 
respectively - 

10 

Disc: Allows the user to draw a disc. To draw a disc 

in the shape of a true circle, the user holds the CTRL key 
down while drawing. 

15 Filled Disk: Allows the user to draw a disc filled 

with solid color. To draw a filled disk in the shape of a 
true circle, the user holds the CTRL key down while drawing. 

20 Adjust Color: Adjusts the existing color. A dialog box 

is displayed on the video monitor 906/908 which allows the 
user a choice of whether to adjust the values of RGB (red, 
green, blue) , HSL {hue, saturation, lightness) , or HSC (hue, 
saturation, contrast) . Further choices are provided based on 

25 the responses to the above determinations, ^Adjust Color" 

applies cmy color manipulations defined in the dialog box as 
the left mouse button is held down and the mouse is dragged 
across the background image. The color adjustments are 
applied evenly, and not on a gradient. 

30 

Grab Color: Allows selectively changing the 

foreground, middle ground, or backgroxind color to a color from 
the image . 



35 



"Special Effects" Group of the Brush Options 

Shuffle: Shuffles the pixels (smallest units of color) 
of the background as the user holds down the left mouse button 
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and drags the mouse across the background image to create a 
speckled effect . This option will be further described 
hereinbelow. 



5 Object Texture: Paints randomly picked colors from 

the current object as the user holds down the left mouse 
button and drags the mouse to create a textured effect , This 
option will be further described hereinbelow. 

10 Object Stamp : Clones the current object as the user 

holds down the left mouse button and drags the mouse to create 
a wallpaper effect. This option works best with a large shape 
brush . 



15 Stamp Re-Color: Combines the effects of "Stamp" and 

"Re-Color". This option clones the current object as the user 
holds down the left mouse button and drags the mouse, while 
re- coloring the cloned objects, 

20 Re-Color: Creates an effect known as "Ted Turner 

colorization" . Maintains the lightness of the background 
while changing the hue and saturation to that of the user- 
selected foreground, middle ground, or background color, 
depending on whether the user holds down and drags with the 

2 5 left, middle, or right mouse button, respectively. 

Posterize: Reduces the number of colors as the user 

holds down the left mouse button and drags the mouse to create 
a poster effect. 



30 



Gray-Scale: Transforms to gray-scale an area over 

which the user drag the mouse while holding down the left 
mouse button. This effect is achieved by averaging the reds, 
greens, and blues. 



35 



Smudge : Smudges the colors of the background as the 
user holds down the left mouse . button and drags the mouse . 
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This option drags particular pixels of color across the 
background . 

Zoom: Allows the user to zoom in on the image to 

5 assist in detailed manipulations. Alternatively, the user may 

zoom out from the image. 

Detail of Shuffle, Object Texture, and Object Stamp Options 
The "Shuffle", "Object Texture", and "Object Stamp" 

10 options are now further described and an example of each is 

given. "Shuffle" is described first. 

Shuffle provides the capability of shuffling or mixing 
the pixels of the background to create a speckled effect. The 
user defines an area of a background image and then grabs the 

15 defined area as an object. The area defining and grabbing 

steps uniquely identify each pixel and the corresponding 
flags, color, tags, and other attributes of each pixel of the 
object. When the user draws somewhere else on the background 
image, using the object pixels as the brush, the computer 902 

20 randomizes or shuffles the pixels to re-color whatever was 

there previously. The randomization is done by the computer 
902 using an array, such that no two pixels which were 
originally adjacent in the image are adjacent after the 
shuffle option is performed. The pixels of the object, with 

25 all their corresponding attributes, are randomly used to re- 

color the background image. 

For example, if a backgroimd image had a beautiful sunset 
sky, but also included an airplane, the user might want to 
delete the airplane. Traditional methods of removing the 

30 airplane would leave a sharp edge where the airplane was 

previously. Blurring the edge is helpful, but shuffle 
produces better results. In this example, the user defines an 
area of the sky next to the airplane as the object. The user 
then recolors the area of the airplane with a randomized set 

35 of pixels from the adjacent sky area using shuffle to produce 

a seamless background image not having the airplane. 



wo 94/28490 PCTAJS94/05756 

-82- 

"Object Texture" provides the ability to change the 
texture of an area or object by using an object of a different 
texture. Texture is the order of pixels of the image. For 
example, if the background image has a person and a marble 
5 desk, the user may grab a piece of marble as an object. Using 

the artifacts of the marble for the brush, such as the veins 
and colors, the user could change the image of the person to 
appear to be on marble, of marble, or to include both effects. 
In other words, the person would look like he was made of 

10 marble or the picture would look as if the person's image was 

printed on marble. 

"Object Stamp" provides the ability to clone or stamp the 
current object as the user drags the pointing device 924. The 
user clicks the left mouse button and drags the mouse over the 

15 areas of an image that the user wishes to stamp. The result 

is a wallpaper effect on the image. To tmdo a portion of the 
clone, the user clicks or drags with the right mouse button 
over the portion of the image that is to be undone. 

Another use of Object Stamp is for placing a border 

20 around an image using the current object. Referring to 

Figures 24a and 24b, for example, the user can use a filigree 
object or a plain "o" shape object 1480 to create a border 
1478 near the edges of an entire image 1500 using Object 
Stamp. The process to create the border uses one complete 

25 object 14 80 as if it is divided into nine equal pieces 

1482-1498. The object can be considered to be divided into 
three rows and three columns. The portion 1482 of the object 
at the first row and first column is placed at the top left 
comer 1502 of the image, while the portion 1486 of the object 

30 at the first row and third column is placed at the top right 

comer 1504 of the image. The portion 1484 of the object at 
the first row and second column is repeated as necessaary to 
fill in the space between the top comers 1502, 1504 of the 
image. A similar operation is done using portions 1494, 14 96, 

35 1498 of the object at the third row to create the bottom 

border of the image. The portions 1488, 1492 of the object at 
the second row, first and third columns, respectively, are 
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repeated as necessary to fill in the space between the left 
corners 1502, 1506 of the image, and the right corners 1504, 
1508 of the image, respectively. The portion 1490 of the 
object at the second row and second column (the center) is not 
5 used as part of the border 1478, 

The following steps are performed to create the border: 
First, the user selects the object 14 80 that is to be used as 
a border. Second, the user clicks on the Object Stamp option 
with the left mouse button. Third, the user moves the cursor 

10 to a location where the border is to begin. The cursor should 

be placed at the desired distance from the top of the image 
and from the left side of the image. The border 1478 will be 
drawn around the entire image using the specified distances 
from the edges. Fourth, the user clicks the middle mouse 

15 button to draw the border 1478, or the user can hold down the 

CTRL key while clicking the left mouse button. To undo a 
portion of the border, the user clicks or drags with the right 
mouse button over the portion of the image that is to be 
undone . 

20 



Object Options 

The object options step 1052 of Figure 19b includes a 
plurality of options to manipulate objects. Objects must 

25 first be selected before manipulations can be done on the 

object. Selection is accomplished using the pointing device 
924 and any of the area selection methods previously 
described. Then the user clicks on the "Grab Object" option 
to indicate to the system 900 that the selected area is now to 

30 be treated as an independent object. Some of the options that 

the user may wish to perform on an image using a mask are in 
the object options grouping. Object options are summarized as 
follows : 



35 



Trim: Allows the user to trim off any undesired edges 

of an object to reveal the background below. The user 
positions the cursor on the portion of the object desired to 
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be trimmed and clicks the left mouse button. While in trim 
mode, the user can see through the cursor to the background 
beneath the object to help in trimming off the edges of the 
object. If a mistake is made and something is trimmed off 
5 that is not desired to be trimmed, the user clicks the right 

mouse button on the area to be " un- trimmed" . The user can 
selectively adjust the size and shape of the "trimmer" brush. 

Sharpen: Increases the contrast between adjacent colors 
10 to more sharply define the object. A sub-menu is displayed on 

the video monitor 906/908 which allows the user to choose the 
level of sharpening from among low, medium, or high. 

Blur: Blurs the edges between one color and another 

15 to create a soft effect. A sub-menu is displayed which allows 

the user to choose the level of blurring from among low, 
medium, or high. 

Fill: Fills the object with color. A dialog box is 

20 displayed which allows the user to choose whether to fill the 

object evenly or on a gradient. 

Tint: Applies a shade of the foreground color to the 

object. Tint replaces the hue and saturation of the object 
25 with that of the tint applied, while maintaining the lightness 

of the object. 

Posterize: Reduces the number of colors in the 

object. A sub-menu is displayed which allows the user to 
30 choose how many colors the object is reduced to. 

Gray-Scale: Transforms the object to gray-scale by 

averaging the reds, greens, and blues of the object. 

35 Adjust Color: Adjusts the existing color in the object. 

A dialog box is displayed which allows the user a choice of 
whether to adjust the color evenly or on a gradient. Then, a 



wo 94/28490 PCT/US94/05756 

-85- 

choice of whether to adjust the values of RGB (red, green, 
blue) , HSL (hue, saturation, lightness) , or HSC (hue, 
saturation, contrast) is given. Further choices are provided 
based on the responses to the above determinations. 

5 

Blend (an object into the background) : Averages the 
colors of the current object with the colors of the background 
behind it to create a shadow effect. The user presses and 
holds down the left mouse button, drags the mouse across the 
10 areas to be blended, and then releases the mouse button. The 

user can selectively adjust the size and shape of the 
"blender" brush. If a mistake is made and something is 
blended that is not desired to be blended, the user clicks the 
right mouse button on the area to be "un-blended" . 



15 



20 



25 



Adjust Transparency: Adjusts the transparency of the 

object. A dialog box is displayed which allows the user to 
choose whether to adjust the transparency evenly or on a 
gradient . 

Move: Moves the current object over the background 

image and over other objects with lower priority. The move 
can be accomplished by use of the pointing device 924 or with 
the keyboard 922. 



"Size. Rotate, and Distort" Group of Object Options 

This group of object options allows the user to change 
the size, rotate, or perform other related operations on the 
current object, A dialog box is displayed on the video 

30 monitor 906/908 which allows the user to type in the angle at 

which to rotate the object, or to click on "+90 or "-90" to 
rotate the object 90 degrees clockwise ( + ) or counterclockwise 
(-) from the current position. The dialog box also allows the 
user to change the size of the object by typing in a new 

35 horizontal or vertical scale, or a new horizontal or vertical 

pixel count. The system 900 automatically updates the pixel 
count when the user changes the scale, and vice versa. The 
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user can also choose whether or not to keep the original 
proportions of the object. If the user chooses proportional 
sizing, for example, whenever the user changes the horizontal 
scale or pixels, the system 900 automatically changes the 
5 vertical scale and pixels to maintain the object's original 

proportions. The dialog box also displays the horizontal and 
vertical pixel coiint of the current object and the original 
object for comparison. 

10 Flip Horizontal: Flips the current object to a mirror 

image of itself, horizontally . 

Flip Vertical: Flips the current object to a mirror 

image of itself, vertically (upside down) . 

15 

Restore: Restores the object to its original size 

and orientation, thus losing any changes made with the 
"Rotate" or "Size" options, even if the object was saved after 
a rotate or size operation. 

20 

Size: Allows the user to interactively change the 

size of the current object. A gray border with yellow corners 
is displayed around the outside of the object which allows the 
user to pull or push the object to enlarge it or reduce it. 

25 If the user clicks and drags on the gray border at the 

top or the bottom of the object outward, the object is 
elongated. Conversely, if the user clicks on the top or 
bottom gray border and drags it inward, the object is 
shortened. Likewise, the user can make the object wider or 

3 0 narrower by clicking and pulling or pushing on the side gray 

border of the object. To increase or decrease both the height 
and width of the object at once, the user clicks on any of the 
yellow comers around the object and drags the mouse up, down 
or sideways . 



Rotate: Allows the user to rotate the current object. 
A gray border with yellow comers is displayed around the 
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outside of the object which allows the user to rotate the 
object manually to any angle and in any direction (clockwise 
or counterclockwise) desired. The user clicks on a yellow 
comer and drags the mouse in the direction (clockwise or 
5 counterclockwise) the object is to be rotated. The mouse 

button is released by the user when the object has reached the 
desired angle of rotation. The border around the object does 
not rotate . 

10 Distort: Allows the user to distort the current object, 

A gray border with yellow comers is displayed around the 
outside of the object which allows the user to push or pull 
the comer in any direction desired. The user clicks on any 
of the yellow comers of the object and drags the mouse in the 

15 direction desired to be distorted. The user releases the 

mouse button when the object has reached the desired 
distortion. The other three comers of the object remain in 
place. The object is distorted but the border around the 
object is not distorted, 

20 

"Using More than One Object" Group of Object Op tions 

The user can select as many objects as desired. However, 
only one object at a time can be manipulated; this object is 
called the cxirrent object. The following describes how to 
25 switch between objects and other ways of using more than one 

object. 

The user can quickly msike another object become the 
current object simply by clicking inside that object. The 
other methods of using .more than one object require selecting 
30 an option at the object options step 1052 of Figure 19b. 

These methods of the object options are as follows: 

Next Object: Makes the next object become the 
"current" object so the user can manipulate it. The computer 
35 902 cycles through all objects as the user continues to click 

on "Next Object". This option applies only if more than one 
object has been selected and includes hidden objects. 
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Knit Objects : Allows the user to stitch together or 
"knit" several objects from the active document window into 
one object, using the current position of the objects in the 
active document window. . The knitted objects will then no 

5 longer exist as independent objects* This is useful, for 

instance, to combine several independent objects of eyes, 
nose, mouth, and ears to create one object of a face. 

A dialog box is displayed on the video monitor 90 6/908 
having a list of objects ("Current Objects") in the active 

0 document window. The user double-clicks on each object 

desired to be knitted into one resultant object. As the user 
double-clicks on the object, it is removed from the list of 
objects in the active document window and transferred to the 
"Objects to Knit" list. If the user desires to remove an 

5 object before the "OK" button is clicked, a double-click on 

the object in the "Objects to Knit" list moves the object back 
to the "Current Objects" list as an independent object. Once 
an object has been knitted, it ccoinot be made independent 
again . 

0 After the user clicks the "OK" button in the "Knit 

Objects" dialog box, a "Describe Objects" dialog box appears 
on the video monitor 906/908 to allow the user to assign a 
description to the knit object. Just as with an independent 
object, the system 900 assigns a priority level number to the 

5 knit object. The number and abbreviated description appear on 

the lower portion of the video monitor 906/9 08 when the knit 
object is the current object. Attaching a description does 
not save the object. 

0 Hide: Hides the current object from view. The user 

sees a box outlining the location of the hidden object when it 
is the current object. To bring the object back into view, 
the user ensures it is the current object and then clicks on 
"Hide" again. 



Delete: Deletes the current object. 
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Clone Object: Duplicates the current object by making 
a "clone" or exact copy of the current object. The clone 
rests on top of the original object until the user moves it. 

"Layering Objects" Group of Objec t Options 

Each object is assigned a priority level when it is 
created. The most recently created object is assigned the 
highest priority level and the first object created is 
assigned the lowest priority level, one (1) . If two or more 
objects overlap, the object with the highest priority level is 
on top and in full view. The current object's priority level 
number appears on the lower portion of the video monitor 
906/908. Layering was previously described in conjunction 
with Figures 3c, 4h, 11, and 12b. 

Raise: Raises the priority level of the current object 
by one level in relation to other objects. 

Lower: Lowers the priority level of the current object 
20 by one level in relation to other objects. 

Delete: Deletes the current object. 

"Affix an Object to the Background" Group of Object O ptions 
25 Affix Copy: Pastes a copy of the object onto the 

image background, so that it becomes part of the background. 
Once an .object copy is affixed to the background, the copy is 
no longer an independent object. The original object remains 
on top of the newly affixed object. The user may continue to 
30 move and manipulate the object or delete it. 

Affix a Copy Blurred: Pastes a copy of the object 

onto the background just as "Affix Copy" does, but softens or 
blurs the edges of the object so it blends smoothly into the 
35 background. 



10 
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Affix All Objects: Pastes a copy of all objects in the 
active or current image window, including hidden objects, onto 
the image backgroxind. The original objects remain on top of 
the object copies until deleted by the user. The affixed 
5 objects ' are no longer objects but become part of the 

background. An option to blur the edges of the objects as 
part of the affix operation is available. 

Affix and Delete All Objects: Pastes a copy of all 
objects, including hidden objects, onto the background and 
deletes the original objects. The affixed objects are no 
longer objects but become part of the background. An option 
to blur the edges of the objects as part of the affix 
operation is available. 

Lock/Unlock: The user may desire to "lock in" all the 
characteristics of an object without actually pasting it to 
the backgroxind. To do so, the user ensures the object is the 
current object, and then clicks on the "Ijock/Unlock" object 
option. This prevents the user from moving, trimming, sizing, 
rotating, distorting, or changing the colors of the object. 
The lower portion of the video monitor 906/908 indicates that 
the object is locked. Once an object is locked, the user can 
easily unlock it by clicking on "Lock/Unlock" again. The user 
can then again move, size, rotate, distort or change the 
colors of the object. This feature applies to graphical 
objects only, not text objects. 

Text Pattern 

30 " A current object can be used as a text pattern. The 

object can be used as the text foreground, middleground, or 
background. The text pattern feature is most noticeable when 
a large size font is used. Text pattern uses the current 
object, e.g., the user's face> in one of four pattern formats: 

35 Tile, Texture, Stretch to Fill Letter, or Stretch to Fill 

Object, The following steps are performed to use an object as 
a text pattern: 



15 



20 
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1. The user chooses Add from a Text menu. 

2. The user places the cursor at the location desired 
to add text and clicks the left mouse button. A 

5 font dialog box appears on the video monitor 

906/908 wherein the bottom of the box has a display 
of the current foregroxind, middleground, and 
background text colors. 

10 3 . The user selects a desired font size and font 

style . 

4 . The user moves the cursor to the text color 
(foreground, middleground, or backgroxind) desired 

15 to be changed to a pattern. 

5. The user clicks the left mouse button while holding 
the SHIFT key. The Select Object dialog box 
appears on the video monitor 906/908 . 

20 

6. The user selects a desired object to be used and a 
text pattern format. The user clicks on the 
desired object in the dialog box from a list of 
objects that are in the active image window . 

25 ' 

The pattern formats are described as follows: 

Tile: The selected object is repeated in the same 

sequence within each letter. If the object is 
3 0 larger than the letter, the entire object is 

not shown. The next letter displays the 
object from the beginning. 

Texture: The selected object is used to create a 
35 textured effect. The pixels of the object 

appear in a random order within each letter. 

Stretch to Fill Letter: The entire selected object is 
used to fill each letter. If the object is 

40 smaller than a letter, the object is stretched 

until it fills the letter. If the object is 
larger than a letter, the object is reduced so 
that the entire object fills the letter. An 
object is never repeated within the same 

45 letter. 

Stretch to Fill Object: The complete selected object is 
used to fill the entire text object so that 
the selected object stretches across all the 
50 letters within the text object. The selected 

object is used only one time. 



Masking Example 
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Ref erring to Figures 25a and 25b, a simple example of 
masking and area options will now be discussed- An original 
image 1350 is of an autumn outdoor scene having five trees. 
Three trees 1352, 1354, 1356 have yellow leaves and two trees 
5 1360, 1362 have green leaves. The user of the system 900 

desires to convey a different look by changing the color of 
only the yellow leaves to red leaves. This change is readily 
done by use of a mask. The user selects an area 13 70 of the 
image that has only the yellow leaves to isolate the yellow 

10 color and create the mask for that color, and sets the "Turn 

Mask On" option. Then, the user selects the "Adjust Color" 
area option to choose the red color to replace the yellow 
color. The resultant image 138 0 still has two trees 1360, 
1362 with green leaves, but now has three trees 1352, 1354, 

15 1356 with red leaves. The rest of the image 13 50, such as 

branches of the trees visible among the leaves, did not change 
as a consequence of the mask and adjust color operations. Of 
course, any of the other options previously listed in the 
discussion of masking could be done in place of the Adjust 

20 Color option to produce a different effect. 

The software described herein for the image editing 
system 900 is written in the "C" language and was translated 
from source code to machine -readsdDle object code using a "C" 
25 version 7.0 compiler available from Microsoft Corporation. 

Nonetheless, one skilled in the technology will recognize that 
the steps in the accompanying flow diagrams can be implemented 
by using a number of different compilers and/or programming 
languages . 

30 The image editing system 100 described herein finds 

application in many environments, and is readily adaptable for 
use therein. For example, the system findsjuse in generation 
of composite photographs where portions of a person's image 
may replace those of another person's image, such as on a 

35 sports team photo. It can be used for production of postcards 

depicting a person in an exotic location. It also can be used 
in applications such as production of driver' s licenses or 
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security access cards where a person's image is combined with 
selected other information on a card. Using the many options 
that include a full range of area color, object manipulation, 
object color correction, brush and text creation tools and the 
5 masking capability, the image editing system 900 can be used 

in many other custom applications. One application is in the 
medical imaging area. Patient images are input via scanners 
or live input and then can be manipulated and layered as 
necessary by medical professionals. Another application is in 

10 the training field, which may include defense training. The 

system allows rapid image capture with automatic backgroxind 
removal for various types of input images. These images may 
then be manipulated, enhanced, and if desired, frame -by -frame 
video hard-copy output provided to enhance the training 

15 experience. Accordingly, the claims are to be interpreted to 

encompass these and other applications of the invention within 
their scope and are not to be limited to the embodiments 
described herein. 

While the above detailed description has shown, described 

20 and pointed out the fundamental novel features of the 

invention as applied to various embodiments, it will be 
iHiderstood that various omissions and substitutions and 
changes in the form and details of the device illustrated may 
be made by those skilled in the art, without departing from 

25 the spirit of the invention. 
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I CliAIM r 

1. In a computer system having a memory, a method of editing 
an image having a plurality of pixels, each pixel having 
a color, comprising the steps of: 

5 selecting a plurality of colors from among those 

present in the image to form a set of selected colors; 
and 

manipulating only those pixels having a color 
defined in the set of selected colors so as to change a 
10 visual feature of the image. 

2. The method as defined in Claim 1, wherein the step of 
selecting a plurality of colors comprises the steps of: 

selecting a first single pixel of the image to 
15 identify a first color; and 

selecting a second single pixel of the image, of a 
different color than the first pixel, so as to identify 
a second color. 



20 3. The method as defined in Claim 2, wherein the step of 

selecting a single pixel includes the step of using a 
pointing device connected to the computer system to mark 
one of the pixels. 

25 4. The method as defined in Claim 1, wherein the step of 

selecting a plurality of colors includes the step of 
selecting a plurality of pixels of the image to form a 
set of selected pixels that identify the plurality of 
colors . 



30 



35 



The method as defined in Claim 4, wherein the step of 
selecting a plurality of pixels includes the step of 
using a pointing device connected to the computer system 
to draw a boundary aroxind the pixels. 

The method as defined in Claim 5, wherein the boundary 
comprises a polygon. 
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7. The method as defined in Claim 4, wherein the step of 
selecting a plurality of pixels includes the step of 
using the pointing device as a brush to electronically 
touch the plurality of pixels. 

5 

8. The method as defined in Claim 4, wherein the step of 
selecting a plurality of pixels includes the step of 
selecting pixels of the image mutually exclusive to the 
set of selected pixels to form a new set of selected 

10 colors. 

9. The method as defined in Claim 1, further comprising the 
step of storing data representing the set of selected 
colors in a list on the memory of the computer system. 

15 

10. The method as defined in Claim 9, wherein the list is 
initially empty. 

11. The method as defined in Claim 9, wherein the list 
20 initially includes all the colors of the image. 

12. The method as defined in Claim 9, wherein the list 
initially includes at least one of the colors of the 
image . 

25 

13. The method as defined in Claim 9, wherein the list is 
structured to link the colors. 

14. The method as defined in Claim 13, wherein the step of 
30 manipulating pixels includes the steps of: 

processing a set of pixels of a color corresponding 
to a first color in the list; and 

processing a set of pixels of another color 
corresponding to a second color linked to the first color 
35 in the list. 
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The method as defined in Claim 9, further comprising the 
step of adding another color to the list by the steps of: 

selecting a different color of the image not in the 
set of selected colors; and 

storing data representing the different color in the 

list. 

The method as defined in Claim 15, wherein the list is 
structured to link the colors. 

The method as defined in Claim 9, additionally comprising 
the step of deleting a color that is stored in the list. 

The method as defined in Claim 17, wherein the step of 
deleting a color further comprises the steps of: 

identifying a color of the image that is stored in 
the list; and 

deleting the identified color from the list. 

The method as defined in Claim 17, wherein the list is 
structured to link the colors . 

The method as defined in Claim 1, wherein the step of 
selecting a plurality of colors further comprises the 
steps of : 

selecting a plurality of pixels of the image that 
identify a single color; and 

repeating the step of selecting a plurality of 
pixels for a different color so as to identify a 
plurality of colors. 

The method as defined in Claim 20, wherein the step of 
selecting a plurality of pixels further includes the step 
of using the pointing device to draw a boundary around 
the pixels. 
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22. The method as defined in Claim l, wherein following the 
step of selecting a plxxrality of colors, the method 
conprises the step of selecting an area within the image 
that is smaller than the image. 

5 

23. The method as defined in Claim 22, wherein the step of 
manipulating pixels comprises the step of changing the 
color value of one or more pixels in the selected area in 
response to the colors defined in the set of selected 

10 colors. 

24. The method as defined in Claim 1, wherein the image 
comprises at least one object. 

15 25. The method as defined in Claim 24, wherein the image 

comprises a plurality of objects and further wherein 
following the step of selecting a plurality of colors, 
the method comprises the step of selecting one of the 
objects. 

20 

26- The method as defined in Claim 25, wherein the image has 
a plurality of X-Y planes located along a Z-axis and the 
selected object is on one of the X-Y planes. 

25 27. The method as defined in Claim 25, wherein the step of 

manipulating pixels comprises the step of changing the 
color value of one or more pixels in the selected object 
in response to the colors defined in the set of selected 
colors . 

30 

28. The method as defined in Claim 1, further comprising the 
steps of: 

providing another image having a plurality of 
pixels; and 

35 repeating the step of manipulating pixels on the new 

image . 
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29. The method as defined in Claim 28, wherein the new image 
is one of a sequence of related images . 

30. The method as defined in Claim 1, wherein the step of 
5 manipulating pixels comprises the step of selecting an 

option to be performed by the computer, and wherein the 
option is selected from a group consisting of: area fill; 
area sharpen; area blur; area adjust color; area tint; 
area posterize; area gray-scale; area stamp-into- 
10 selection; object fill; object sharpen; object blur; 

object adjust color; object tint; object posterize; 
object gray-scale; and brush options* 

31. The method as defined in Claim 1, wherein each color is 
15 different from another color by at least one attribute. 

32- The method as defined in Claim 31, wherein the attributes 
comprise red, green, blue. 

'20 33. The method as defined in Claim 31, wherein the attributes 

comprise hue, saturation, lightness. 

34. The method as defined in Claim 31, wherein the attributes 
comprise hue, saturation, contrast. 



25 



30 



35. The method as defined in Claim 1, wherein the colors 
include substantially all visible colors. 

36- The method as defined in Claim 35, wherein all visible 
colors are represented by a plurality of hues of gray 
including the absolutes of black and white. 



37. In a computer system having a memory, a method of editing 
an image having a plurality of pixels, each pixel having 
35 a color, comprising the steps of: 

identifying a color of each pixel of the image; 
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storing data representing each identified color in 
an original first list on the memory of the computer 

systems- 
identifying a selected color of the image; 
5 deleting the data representing the selected color 

from the first list; and 

manipulating pixels of the image having colors of a 
preselected relationship to the colors identified in the 
first list so as to change a visual feature of the image. 

10 

38. The method as defined in Claim 37, wherein the 
preselected relationship is all colors represented by 
data in the first list. 

15 39. The method as defined in Claim 37, wherein the 

preselected relationship is all colors represented by 
data that has been deleted from the first list. 

40. The method as defined in Claim 37, wherein the step of 
20 identifying a selected color comprises the step of 

selecting a single pixel of the image to identify a 
single color. 

41. The method as defined in Claim 40, wherein the step of 
25 selecting a single pixel comprises the step of using a 

pointing device connected to the computer system to mark 
the pixel. 



30 



42. The method as defined in Claim 40, wherein the step of 
selecting a single pixel is repeated for a different 
color so as to identify a plurality of colors. 



43. The method as defined in Claim 37, wherein the step of 
identifying a selected color comprises the step of 
3 5 selecting a plurality of pixels of the image to form a 

set of selected pixels that identify a plurality of 
colors . 
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The method as defined in- Claim 43, wherein the step of 
identifying a selected color further comprises the step 
of using a pointing device connected to the computer 
system to draw a boundary around the pixels. 

The method as defined in Claim 44, wherein the boundary 
comprises a polygon. 

The method as defined in Claim 43, wherein the step of 
identifying a selected color further comprises the step 
of using a pointing device connected to the computer 
system as a birush to electronically touch the plurality 
of pixels. 

The method as defined in Claim 37, wherein the first list 
is structured to link the colors. 

The method as defined in Claim 47, wherein the step of 
manipulating pixels of the image further comprises the 
steps of : 

processing a set of pixels of a color corresponding 
to a first color in the first list; and 

processing a set of pixels of another color 
corresponding to a second color linked to the first color 
in the first list. 

The method as defined in Claim 37, additionally 
comprising the step of storing data representing the 
selected color in a second list on the memory of the 
computer system; 

The method as defined in Claim 49, wherein the step of 
manipulating pixels of the image is performed on the 
pixels of the image having the colors corresponding to 
the colors identified in the second list. 
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51. The method as defined in Claim 49, wherein the second 
list is initially empty- 

52. The method as defined in Claim 49, wherein the second 
5 list initially includes at least one of the colors of the 

image . 

53- The method as defined in Claim 49, wherein the second 
list is structured to link the colors. 

10 

54. The method as defined in Claim 53, wherein the step of 
manipulating pixels of the image further comprises the 
steps of: 

processing a set of pixels of a color corresponding 
15 to a first color in the second list; and 

processing a set of pixels of another color 
corresponding to a second color linked to the first color 
in the second list. 

20 55. The method as defined in Claim 49, further comprising the 

step of adding data representing another color to the 

second list by the steps of: 

selecting a different color of the image than was 

previously represented in the second list; and 
2 5 storing data representing the different color in the 

second list. 

56. The method as defined in Claim 55, wherein the second 
list is structured to link the colors. 



30 



57. The method as defined in Claim 49, further comprising the 
step of deleting data representing a selected color that 
is stored in the second list. 



35 



58. 



The method as defined in Claim 57, wherein the step of 
deleting data representing a selected color further 
comprises the steps of: 
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identifying a color of the image for which data 
representing such color is stored in the second list; and 

deleting data representing the identified color from 
the second list. 

5 

59. The method as defined in Claim 57, wherein the second 
list is structured to link the colors. 

60. The method as defined in Claim 37, wherein the step of 
10 identifying a selected color comprises the step of 

selecting a plurality of pixels of the image that 
identify a single color. 

61. The method as defined in Claim 60, wherein the step of 
15 selecting a plurality of pixels further comprises the 

step of using a pointing device connected to the computer 
system to draw a boundary around the pixels. 

62. The method as defined in Claim 3 7, wherein following the 
20 step of deleting the data from the first list, the method 

comprises the step of selecting an area within the image 
that is smaller than the image. 

63. The method as defined in Claim 62, wherein the step of 
25 manipulating pixels of the image further comprises the 

step of changing the color value of one or more pixels in 
the selected area in response to the colors identified in 
the first list. 

3 0 64. The method as defined in Claim 62, wherein the step of 

manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected area in response to the colors identified in 
the second list. 



35 



65. The method as defined in Claim 37, wherein the image 
comprises at least one object. 
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The method as defined in Claim 65, wherein the image 
comprises a plurality of objects and further wherein 
following the step of deleting the data from the first 
list, the method conprises the step of selecting one of 
the objects. 

The method as defined in Claim 66, wherein the image has 
a plurality of X-Y planes located along a Z-axis and the 
selected object is on one of the X-Y planes. 

The method as defined in Claim 66, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected object in response to the colors identified 
in the first list. 

The method as defined in Claim 66, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected object in response to the colors identified 
in the second list. 

The method as defined in Claim 37, further comprising the 
steps of: 

providing another image having a plurality of 
pixels; and 

repeating the step of manipulating pixels on the new 
image . 

The method as defined in Claim 70, wherein the new image 
is one of a sequence of related images. 

The method as defined in Claim 37, wherein the step of 
manipulating pixels of the image further comprises the 
step of selecting an option to be performed by the 
computer, wherein the option is selected from a group 
consisting of: area fill; area sharpen; area blur; area 
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adjust color; area tint; area posterize; area gray- scale; 
area stamp-into-selection; object fill; object sharpen; 
object blur; object adjust color; object tint; object 
posterize; object gray- scale; and brush options. 

73 . In a computer system having a memory, a method of 
creating a color mask for an image, comprising the steps 
of: 

providing an image having a plurality of pixels, 
each pixel having a color; 

selecting a plurality of colors from among those 
present in the image to form a set of selected colors; 
and 

storing data representing the colors from the set of 
selected colors in a list on the memory of the computer 
system, to thereby define the mask. 

74. The method as defined in Claim 73, further comprising the 
step of manipulating only those pixels of the image 
having colors corresponding to the colors represented in 
the mask so as to change a visual feature of the image. 

75. The method as defined in Claim 74, wherein following the 
step of storing data in a list, the method comprises the 
step of selecting an area within the image that is 
smaller thcui the image. 

76. The method as defined in Claim 75, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected area in response to the colors represented 
in the mask. 

77. The method as defined in Claim 74, wherein the image 
comprises at least one object. 
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78. The method as defined in Claim 77, wherein the image 
comprises a plurality of objects and further wherein 
following the step of storing data in a list, the method 
comprises the step of selecting one of the objects. 

5 

79. The method as defined in Claim 78, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected object in response to the colors represented 

10 in the mask. 

80. In a computer system having a memory, a method of 
creating an inverted color mask for an image, comprising 
the steps of : 

15 providing an image having a plurality of pixels, 

each pixel having a color; 

identifying the color of each pixel of the image to 
form a set of identified colors; 

selecting a color of the image; and 
20 deleting the selected color from the colors in the 

set of identified colors, thereby defining an inverted 
mask. 

81. The method as defined in Claim 80, further comprising the 
25 step of storing data representing each color from the set 

of identified colors in a first list on the memory of the 
coit^uter system. 

82. The method as defined in Claim 80, further comprising the 
3 0 step of storing data representing the selected color in 

a second list on the memory. 

83. The method as defined in Claim 82, wherein the second 
list defines a noninverted mask. 

35 

84. The method as defined in Claim 83, additionally 
comprising the step of manipulating pixels of the image 
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having the colors corresponding to the colors identified 
in the noninverted mask so as to change a visual feature 
of the image . 

5 85. The method as defined in Claim 84, wherein the image 

comprises at least one object, 

86, The method as defined in Claim 85, wherein the image 
comprises a plurality of objects and further wherein 

10 following the step of deleting the selected color, the 

method comprises the step of selecting one of the 
objects . 

87. The method as defined in Claim 86, wherein the step of 
15 manipulating pixels of the image further comprises the 

step of changing the color value of one or more pixels in 
the selected object in response to the colors represented 
in the noninverted mask. 

20 88. The method as defined in Claim 84, wherein following the 

step of deleting the selected color, the method comprises 
the step of selecting an area within the image that is 
smaller than the image. 

25 89. The method as defined in Claim 88, wherein the step of 

manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected area in response to the colors represented 
in the noninverted mask. 

30 

90. The method as defined in Claim 80, additionally 
comprising the step of manipulating pixels of the image 
having colors corresponding to the colors represented in 
the inverted mask so as to change a visual feature of the 
3 5 image . 
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91. The method as defined in Claim 90, wherein following the 
step of deleting the selected color, the method comprises 
the step of selecting an area within the image that is 
smaller than the image . 

5 

92- The method as defined in Claim 91, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected area in response to the colors represented 
10 in the inverted mask. 



93. The method as defined in Claim 90, wherein the image 
comprises at least one object. 



15 94. The method as defined in Claim 93, wherein the image 

comprises a plurality of objects and further wherein 
following the step of deleting the selected color, the 
method comprises the step of selecting one of the 
objects. 

20 

95. The method as defined in Claim 94, wherein the step of 
manipulating pixels of the image further comprises the 
step of changing the color value of one or more pixels in 
the selected object in response to the colors represented 
25 in the inverted mask. 



96. A system for editing an image having a plurality of 
pixels, each pixel having a color, comprising: 
a computer having a memory; 
30 means for providing the image; 

means for identifying a color of each pixel of the 
image ; 

means for storing data representing each identified 
color in a first list on the memoiry of the computer; 
35 means for identifying a selected color of the image; 

means for deleting the data representing the 
selected color from the first list; and 
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means for manipulating pixels of the image having 
colors corresponding to the colors identified in the 
first list so as to change a visual feature of the image. 

5 97 . The system as defined in Claim 96 , further comprising 

means for storing data representing the selected color in 
a second list on the memory. 

98. The system as defined in Claim 97, further comprising 
10 means for manipulating pixels of the image having colors 

corresponding to the colors identified in the second list 
so as to change a visual feature of the image. 

99. In a computer system, a method of recoloring an object 
15 having a plurality of pixels, comprising the steps of: 

providing a background image having a plurality of 
pixels; 

selecting an area from the background image; 

determining a hue of a selected pixel in the 
20 selected area; 

repeating the determining step xintil the hue of all 
pixels in the selected area has been determined; 

determining an average hue of all the pixels in the 
selected area; 

25 selecting an object to be recolored; and 

changing the hue of the selected object to the 
calculated average hue. 

100. The method as defined in Claim 99, wherein the selected 
30 - area is a single pixel. 

101- The method as defined in Claim 99, wherein the step of 
selecting an object includes the step of selecting a tool 
defining a brush with which to recolor the selected 
35 object. 
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102. In a computer system, a method of recoloring an object 
having a plurality of pixels, comprising the steps of: 

providing a background image having a plurality of 
pixels; 

5 selecting an area from the background image; 

identifying the selected area as a first object; 
determining a hue of a selected pixel of the first 
object; 

repeating the determining step until the hue of all 
10 pixels of the first object has been determined; 

determining an average hue of all the pixels of the 
first object; 

identifying a second object to be recolored; and 

changing the hue of the second object to the average 
15 hue of the first object. 
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